hockeypuck-dev team mailing list archive

[Casey Marshall <cmars@xxxxxxxxxxxxx>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

At lp:hockeypuck, rev 212, I've addressed inconsistencies between SKS
and Hockeypuck key parsing and storage.

Hockeypuck still validates key material when it can. When it can't, it
stores the raw packet data for hashing consistency with SKS.
Eventually the validation result will be stored in the STATE column,
to filter such material from HKP responses. SKS already seems to do
this to a limited extent. For example (and this was surprising to me,
when I discovered it), I can add arbitrary UID packets to anyone's key
material, and SKS will store and include them in the hash calculation
for recon. However, SKS will filter out such packets on op=index,
op=get, etc.

Yesterday I loaded Hockeypuck and an SKS 1.1.4 instance with a single
SKS dump file containing 15000 keys. Reconciliation was successful (no
repeated requests for missing keys)! I'm currently loading a full SKS
dump into my Hockeypuck and SKS instances, to try another recon test
at scale.

There are still some performance issues. Bulk loading the openpgp_*
tables is better, but now the PostgreSQL prefix tree implementation
is the bottleneck. On each insert, the polynomial result for each
parent node has to be updated. And then there's splitting and joining --
lots of queries mixed with inserts and updates. Putting all of them in
a single transaction seems to break the consistency of the queries,
when I've tried this, the evaluation points came out wrong. It seems
like a simpler key-value store would be better here... I'm considering
diskv.

- -Casey
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJSP6IDAAoJEIec+KqN2jAaxEQP+weXOALd6XPif5QMLdkLom1d
8mmI55TL8dzthsBdyskaNxQqjNJLawG2jCsv6HBi9tufgwVZ8NJyzWH/tSLKd44i
ei9DHZZ182oxatf8amKPD27qC7HFzZJzcuO/Yj54plXxW4rL0AW8+VKDLrS28X2J
RCzvu/RUxRY0/eOAorKUU38VQDiiWZlZloKePA6cyoDe8Fu6fJaSnWscpDCq7nK+
+Z3WdhbnOCZWSPewZBhKUE6fBPC98winqhy9YzsdHVPEl9jbnsuQrj9oeooSvjLp
XThxvGFgtmnuwYag/sJW38jgPlCEZ938YYI/YrpVsgYrVufoY/nQM0Lgl7RrOR5u
iKDCog/ZFYgRZ/fWT5iub4Tui1Il4nLepyPOD1J1r+kJAavFxLuPRbiOiUakbDsg
XHdvyqJ6WVvbYcr0lM14yq4EexOUeKmzywuaA6LfPh63+qDZ9QecHVzXKsVrVd6i
OxwF/btF5LyWL8ygZjyaXZfbMvBtOvQI5T4w+MT0KR6WMPb1M37waR7GqwGrg/ht
TMmUZMIpGYImfEngZIvY4Vq2JocW++JWrjSg0JtKLu/CTMfw+DNfVp1FnTK492Tv
5WbvUZ6xv7CeEHyMDVN7FOo08of8uU+HXe144bqLZMFhLMEo6R8hjuDVKHm8EjZn
eog/1ysaChsL7Wrh7LQj
=kDl+
-----END PGP SIGNATURE-----