Rust-Parity db: ParityDB — Fast and reliable database, optimised for read operation

A database for the blockchain.

Design considerations

API

The database is a universal key-value storage that supports transactions. It does not support iteration or prefix-based retrieval.

State-optimized

90% Of blockchain data and IO is trie nodes. Database should allow for efficient storage and retrieval of state data first.

Single writer

Database should be able to support multiple concurrent readers. It is sufficient to allow a single concurrent writer.

No cache

Low level LRU caching of blockchain data, such as individual trie nodes, proves to be inefficient. Cache should be done on a higher level of abstractions. I.e. storage items or block headers.

Transaction isolation

Transaction are applied atomically. Queries can't retrieve partially committed data.

Durability

Database should be restored to consistent state if IO is interrupted at any point.

Implementation

Data structure

Data is organized into columns. Each column serving a particular type of data, e.g. state or headers. Column consists of an index and a set of 16 value tables for varying value size.

Index

Index is an is mmap-backed dynamically sized probing hash table. Each entry is a page of 64 8-byte entries, making 512 bytes. Each 64-bit entry contains 32 bits of value address, 4 bits of value table index and 28 bit value c derived from k. c is computed by skipping n high bits of k and taking the next 28 bits. k is 256-bit key that is derived from the original key and is uniformly distributed. n is current index bit-size. First n bits of k map k to a page. Entries inside the page are unsorted. Empty entry is denoted with a zero value. Empty database starts with n = 16, which allows to put just 240 bits of k in the value table.

Value tables

Value table is linear array of fixed-size entries that can grow as necessary. Each entry may contain one of the following:

  • Filled entry, that contains 240 bits of k, 16 bit data value size and the actual value
  • Tombstone entry. This contains an index of the previous tombstone, forming a linked list of available entries.
  • Multipart entry. This is much like Filled, additionally holding an address of the next entry that holds continuation of the data.

15 of 16 value tables only allow values up to entry size. An additional table with 8kb entry size is designated for large values and allows multipart entries.

Operations

Lookup

Compute k, find index page using first n bits. Search for a matching entry that has matching c bits. Use the address in the entry to query the partial k and value from a value table. Confirm that k is indeed what is expected.

Insertion

If an insertion is attempted into a full index page a reindex is triggered. Page size of 64 index entries trigger a reindex once load factor reaches about 50%.

Reindex

When a collision can't be resolved, a new table is created with twice the capacity. Insertion is immediately continued to the new table. A background process is started that moves entries from the old table to the new. All queries during that process check both tables.

Transaction pipeline

On commit all data is first moved to an in-memory overlay, making it available for queries. The commit is then added to the commit queue. This allows for commit function to return as early as possible. Commit queue is processed by a commit worker that collects data that would be modified in the tables and writes it to the available log file. All modified index and value table pages are placed in the in-memory overlay. The file is then handled to another background thread that flushes it to disk and adds it to the finalization queue. Finally, another thread handles the finalization queue. It reads the file and applies all changes to the tables, clearing the page overlay.

On startup if the log files exists they are validated for corruption and enacted upon the tables.

Potential issues

  • Memory mapped IO won't be able to support 32-bit systems once the index grows to 2GB.
  • Size amplification. Index grow up to about 50% capacity before rebalance is triggered. Which means about 50% of allocated space is actually used for occupied index entries. Additionally, each value table entry is only partially filled with actual data.

Comments

  • Warn about database reaching its limits
    Warn about database reaching its limits

    Apr 15, 2020

    Print a warning if any of the internal hard limits is at 50% capacity. This includes:

    • 32-bit value table entry address space
    • fields of the 64-bit index entry Also detect and complain about index degradation due to high collision rate.
    Reply
  • Very slow on spinning disks
    Very slow on spinning disks

    May 11, 2020

    Syncing on HDDs is very slow (much slower than RocksDB). This might be a “won’t fix” but if so it should be documented.

    Reply
  • io_uring
    io_uring

    Jul 8, 2020

    This is a significant performance win.

    Reply
  • Btree Indexing
    Btree Indexing

    Dec 21, 2021

    This PR contains my initial indexing code. I create a draft PR as things starts to work properly and I am looking a bit more at the performances, but welcome initial feedback. The PR is a bit overly complex, but there is some perf/design I wanted to test so I added some abstraction to check that (not done yet).

    Also I did use a lot of code from a different branch which explain I started with a separate file for the btree node storage. I wanted constant size node storage, which I am not convinced is interesting: Storage of big key is pretty costy (key is stored with value :(). So storing in value tables (or its own value tables) may end up being better (untested yet).

    Also this expose a specific api: one column we can have both hash indexed and btree indexed contents. Specializing column to either hash indexed or btree indexed can be an option too (I could restore some migration optimization then).

    • in column I did split write plan, there is still some code from a different branch where locked are used as parameter, not strictly needed. It is very slightly better for not recursively searching index when reindexing, but that is not really relevant.

    • there is a race in tests, deadlock sometimes. I think it is from the logic of the db test (see TODO linked pr).

    • key as trait is also something from a different branch, it could be replaced by just be an enum, or make specialized table function (for_parts is containing lot of logic).

    Main open questions at this point of the PR:

    • switch to column specialized for indexed content model?
    • switch to variable size nodes stored in value tables?
    • switch CommitSet back to something similar to master
    Reply
  • Consider using direct I/O
    Consider using direct I/O

    Jul 8, 2020

    Since caching individual trie nodes is inefficient, there may be advantages to bypassing the OS page cache and using our own caches exclusively.

    Reply
  • fix: madvise_random on not(unix)
    fix: madvise_random on not(unix)

    Sep 9, 2021

    should return () instead of Result<()>

    Reply
  • Compression
    Compression

    Apr 28, 2021

    @arkpar , I did continue a bit your compression branch. I am opening this PR with stats I obtained, this is a slimdown version of my working branch https://github.com/paritytech/parity-db/compare/compression...cheme:compression (this branch is relevant to test quickly existing db compression, but is a bit fat).

    I did test a few algo on the latest polkadot snapshot, snappy and lz4 are the best I found.

    My test show that compressing everything result in a bigger db (~ +200Meg on a _7G db). So the PR uses a flag to allow storing compressed or uncompressed value. I did put the flag in the Rc, reducing max rc by 2 (even if I think removing Rc when inactive should be good), but that is just a possibility. I also switch the behavior when max rc is reached (lock the value in the db). When storing compressed only when there is a size gain, I go from 6.8G to 6.3G (6.2 with slower methods), it did not strike me as much, but seems to be rather costless.

    Thinking twice, it may not be needed to use this flag, just defining

    I also added the possibility to use a minimal size threshold for compression.

    These are some stats from the migration on the latest snapshot of polkadot db with lz4 and storing everything as compressed, I did change the histogram range to avoid increasing meta size:

    Results: Column 0 Total values: 4726042 Total bytes: 191357393 Total oversized values: 0 Total oversized bytes: 0 Total commits: 0 New value insertions: 4726042 Existing value insertions: 0 Removals: 0 Missed removals: 0 Compressed bytes: 191357393 Compression deltas: 0-31: -10 32-63: -27205818 64-95: -995550 96-127: -140976 128-159: -12870 160-191: -966 192-223: -30 224-255: -18 288-319: -7 320-351: -7 Queries per size tier: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] Missed queries: 0 Value histogram: 0-31: 2 32-63: 4534303 64-95: 165925 96-127: 23496 128-159: 2145 160-191: 161 192-223: 5 224-255: 3 288-319: 1 320-351: 1

    Column 1 Total values: 680522 Total bytes: 147607313 Total oversized values: 2 Total oversized bytes: 2303191 Total commits: 0 New value insertions: 680522 Existing value insertions: 0 Removals: 0 Missed removals: 0 Compressed bytes: 147607313 Compression deltas: 32-63: -215755 64-95: -358258 96-127: 11970480 128-159: -162819 160-191: -109670 192-223: -80623 224-255: -43303 256-287: -34218 288-319: -24737 320-351: -18501 352-383: -17811 384-415: -13237 416-447: 469672 448-479: -9415 480-511: -15994 512-543: -70644 544-575: -60728 576-607: -2475 608-639: -3417 640-671: -4513 672-703: -4536 704-735: -3352 736-767: -2056 768-799: -958 800-831: -3180 832-863: -2950 864-895: -3171 896-927: -2564 928-959: -1338 960-991: -1388 992-1023: -2225 1024-1055: -3078 1056-1087: -2875 1088-1119: -2848 1120-1151: -1048 1152-1183: -2133 1184-1215: -2097 1216-1247: -2877 1248-1279: -2242 1280-1311: -1786 1312-1343: -1325 1344-1375: -2414 1376-1407: -2789 1408-1439: -2638 1440-1471: -2509 1472-1503: -794 1504-1535: -1992 1536-1567: -2479 1568-1599: -3215 1600-1631: -2701 1632-1663: -2413 1664-1695: -1660 1696-1727: -2469 1728-1759: -2679 1760-1791: -2508 1792-1823: -3011 1824-1855: -1743 1856-1887: -1907 1888-1919: -2199 1920-1951: -2542 1952-1983: -2461 1984-2015: -1904 2016-2047: -1225 2048-2079: -2544 2080-2111: -2427 2112-2143: -2886 2144-2175: -2666 2176-2207: -1855 2208-2239: -1895 2240-2271: -2425 2272-2303: -2630 2304-2335: -3004 2336-2367: -2318 2368-2399: -1978 2400-2431: -2376 2432-2463: -2701 2464-2495: -3127 2496-2527: -2860 2528-2559: -1917 2560-2591: -2249 2592-2623: -6651 2624-2655: -3090 2656-2687: -2633 2688-2719: -1896 2720-2751: -1755 2752-2783: -2554 2784-2815: -2899 2816-2847: -3364 2848-2879: -2155 2880-2911: -2127 2912-2943: -2704 2944-2975: -3082 2976-3007: -2926 3008-3039: -2626 3040-3071: -2270 3072-3103: -1479 3104-3135: -2736 3136-3167: -3077 3168-3199: -2711 3200-3231: -2764 3232-3263: -1936 3264-3295: -1861 3296-3327: -2692 3328-3359: -3005 3360-3391: -3005 3392-3423: -2307 3424-3455: -1668 3456-3487: -2046 3488-3519: -2625 3520-3551: -2617 3552-3583: -2148 3584-3615: -1535 3616-3647: -2113 3648-3679: -2103 3680-3711: -2078 3712-3743: -2232 3744-3775: -1921 3776-3807: -1580 3808-3839: -2156 3840-3871: -2071 3872-3903: -2180 3904-3935: -2147 3936-3967: -1528 3968-3999: -1886 4000-4031: -1732 4032-4063: -2064 4064-4095: -1703 4096-4127: -1625 4128-4159: -1475 4160-4191: -1496 4192-4223: -1599 4224-4255: -1902 4256-4287: -1647 4288-4319: -1097 4320-4351: -1537 4352-4383: -1369 4384-4415: -1681 4416-4447: -2040 4448-4479: -1558 4480-4511: -989 4512-4543: -1838 4544-4575: -1346 4576-4607: -1423 4608-4639: -1902 4640-4671: -1310 4672-4703: -960 4704-4735: -1428 4736-4767: -1391 4768-4799: -1376 4800-4831: -1498 4832-4863: -975 4864-4895: -1214 4896-4927: -1347 4928-4959: -1143 4960-4991: -1765 4992-5023: -1413 5024-5055: -1590 5056-5087: -16734 5088-5119: -722 5120-5151: -933 5152-5183: -529 5184-5215: -457 5216-5247: -234 5248-5279: -598 5280-5311: -542 5312-5343: -476 5344-5375: -375 5376-5407: -367 5408-5439: -540 5440-5471: -424 5472-5503: -414 5504-5535: -586 5536-5567: -239 5568-5599: -432 5600-5631: -388 5632-5663: -624 5664-5695: -584 5696-5727: -301 5728-5759: -264 5760-5791: -441 5792-5823: -361 5824-5855: -498 5856-5887: -481 5888-5919: -203 5920-5951: -282 5952-5983: -344 5984-6015: -523 6016-6047: -407 6048-6079: -406 6080-6111: -200 6112-6143: -174 6144-6175: -352 6176-6207: -270 6208-6239: -263 6240-6271: -262 6272-6303: -150 6304-6335: -415 6336-6367: -261 6368-6399: -305 6400-6431: -368 6432-6463: -217 6464-6495: -248 6496-6527: -341 6528-6559: -244 6560-6591: -155 6592-6623: -136 6624-6655: -62 6656-6687: -277 6688-6719: -96 6720-6751: -92 6752-6783: -192 6784-6815: -32 6816-6847: -64 6848-6879: -32 6880-6911: -32 6912-6943: -161 6944-6975: -191 6976-7007: -99 7008-7039: -198 7040-7071: -229 7072-7103: -186 7104-7135: -132 7136-7167: -67 7168-7199: -34 7200-7231: -99 7232-7263: -102 7264-7295: -102 7296-7327: -34 7328-7359: -68 7360-7391: -34 7392-7423: -136 7424-7455: -35 7456-7487: -175 7488-7519: -140 7552-7583: -35 7616-7647: -140 7648-7679: -64 7680-7711: -36 7744-7775: -36 7776-7807: -72 7808-7839: -137 7840-7871: -36 7872-7903: -72 7904-7935: -37 7936-7967: -37 7968-7999: -37 8000-8031: -37 8064-8095: -37 8096-8127: -111 8128-8159: -37 8160-8191: -37 8192-8223: -38 8256-8287: -38 8288-8319: -38 8384-8415: -38 8416-8447: -35 8480-8511: -39 8512-8543: -78 8608-8639: -78 8672-8703: -39 8704-8735: -80 8800-8831: -40 8832-8863: -120 8928-8959: -81 8960-8991: -82 9184-9215: -82 9216-9247: -42 9248-9279: -84 9280-9311: -84 9408-9439: -39 9440-9471: -86 9472-9503: -43 9504-9535: -255 9536-9567: -43 9600-9631: -43 9632-9663: -43 9664-9695: -168 9696-9727: -83 9792-9823: -36 10560-10591: -4 10592-10623: 52 10624-10655: 276 10656-10687: 330 10688-10719: 19198 11872-11903: 2656 11904-11935: 1265 21088-21119: 3642 Queries per size tier: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] Missed queries: 0 Value histogram: 32-63: 36482 64-95: 157656 96-127: 317276 128-159: 29243 160-191: 19664 192-223: 14261 224-255: 9642 256-287: 5816 288-319: 4839 320-351: 3868 352-383: 3112 384-415: 2523 416-447: 29432 448-479: 2101 480-511: 2481 512-543: 8899 544-575: 8167 576-607: 357 608-639: 439 640-671: 652 672-703: 574 704-735: 496 736-767: 351 768-799: 160 800-831: 363 832-863: 369 864-895: 378 896-927: 352 928-959: 209 960-991: 220 992-1023: 322 1024-1055: 328 1056-1087: 335 1088-1119: 288 1120-1151: 126 1152-1183: 223 1184-1215: 272 1216-1247: 303 1248-1279: 232 1280-1311: 204 1312-1343: 129 1344-1375: 221 1376-1407: 257 1408-1439: 244 1440-1471: 235 1472-1503: 146 1504-1535: 185 1536-1567: 219 1568-1599: 274 1600-1631: 228 1632-1663: 204 1664-1695: 145 1696-1727: 214 1728-1759: 236 1760-1791: 226 1792-1823: 237 1824-1855: 140 1856-1887: 155 1888-1919: 175 1920-1951: 203 1952-1983: 193 1984-2015: 152 2016-2047: 95 2048-2079: 192 2080-2111: 176 2112-2143: 208 2144-2175: 193 2176-2207: 134 2208-2239: 137 2240-2271: 176 2272-2303: 191 2304-2335: 204 2336-2367: 156 2368-2399: 134 2400-2431: 159 2432-2463: 182 2464-2495: 211 2496-2527: 192 2528-2559: 131 2560-2591: 146 2592-2623: 417 2624-2655: 196 2656-2687: 167 2688-2719: 120 2720-2751: 111 2752-2783: 162 2784-2815: 183 2816-2847: 200 2848-2879: 127 2880-2911: 126 2912-2943: 160 2944-2975: 184 2976-3007: 174 3008-3039: 156 3040-3071: 135 3072-3103: 84 3104-3135: 154 3136-3167: 172 3168-3199: 152 3200-3231: 155 3232-3263: 111 3264-3295: 104 3296-3327: 152 3328-3359: 161 3360-3391: 159 3392-3423: 123 3424-3455: 88 3456-3487: 109 3488-3519: 139 3520-3551: 139 3552-3583: 114 3584-3615: 89 3616-3647: 117 3648-3679: 112 3680-3711: 105 3712-3743: 113 3744-3775: 97 3776-3807: 79 3808-3839: 109 3840-3871: 99 3872-3903: 105 3904-3935: 103 3936-3967: 75 3968-3999: 92 4000-4031: 84 4032-4063: 99 4064-4095: 82 4096-4127: 76 4128-4159: 68 4160-4191: 70 4192-4223: 74 4224-4255: 87 4256-4287: 76 4288-4319: 51 4320-4351: 70 4352-4383: 60 4384-4415: 75 4416-4447: 90 4448-4479: 68 4480-4511: 44 4512-4543: 81 4544-4575: 59 4576-4607: 63 4608-4639: 80 4640-4671: 56 4672-4703: 40 4704-4735: 61 4736-4767: 59 4768-4799: 58 4800-4831: 63 4832-4863: 41 4864-4895: 49 4896-4927: 56 4928-4959: 46 4960-4991: 71 4992-5023: 57 5024-5055: 66 5056-5087: 672 5088-5119: 29 5120-5151: 37 5152-5183: 21 5184-5215: 18 5216-5247: 9 5248-5279: 23 5280-5311: 21 5312-5343: 19 5344-5375: 14 5376-5407: 14 5408-5439: 20 5440-5471: 16 5472-5503: 16 5504-5535: 22 5536-5567: 9 5568-5599: 16 5600-5631: 15 5632-5663: 23 5664-5695: 21 5696-5727: 11 5728-5759: 10 5760-5791: 16 5792-5823: 13 5824-5855: 18 5856-5887: 17 5888-5919: 7 5920-5951: 10 5952-5983: 12 5984-6015: 19 6016-6047: 15 6048-6079: 14 6080-6111: 7 6112-6143: 6 6144-6175: 12 6176-6207: 9 6208-6239: 9 6240-6271: 9 6272-6303: 5 6304-6335: 14 6336-6367: 9 6368-6399: 10 6400-6431: 12 6432-6463: 7 6464-6495: 8 6496-6527: 11 6528-6559: 8 6560-6591: 5 6592-6623: 5 6624-6655: 2 6656-6687: 9 6688-6719: 3 6720-6751: 3 6752-6783: 6 6784-6815: 1 6816-6847: 2 6848-6879: 1 6880-6911: 1 6912-6943: 5 6944-6975: 6 6976-7007: 3 7008-7039: 6 7040-7071: 8 7072-7103: 6 7104-7135: 4 7136-7167: 2 7168-7199: 1 7200-7231: 3 7232-7263: 3 7264-7295: 3 7296-7327: 1 7328-7359: 2 7360-7391: 1 7392-7423: 4 7424-7455: 1 7456-7487: 5 7488-7519: 4 7552-7583: 1 7616-7647: 4 7648-7679: 2 7680-7711: 1 7744-7775: 1 7776-7807: 2 7808-7839: 4 7840-7871: 1 7872-7903: 2 7904-7935: 1 7936-7967: 1 7968-7999: 1 8000-8031: 1 8064-8095: 1 8096-8127: 3 8128-8159: 1 8160-8191: 1 8192-8223: 1 8256-8287: 1 8288-8319: 1 8384-8415: 1 8416-8447: 1 8480-8511: 1 8512-8543: 2 8608-8639: 2 8672-8703: 1 8704-8735: 2 8800-8831: 1 8832-8863: 3 8928-8959: 2 8960-8991: 2 9184-9215: 2 9216-9247: 1 9248-9279: 2 9280-9311: 2 9408-9439: 1 9440-9471: 2 9472-9503: 1 9504-9535: 6 9536-9567: 1 9600-9631: 1 9632-9663: 1 9664-9695: 4 9696-9727: 2 9792-9823: 1 10560-10591: 2 10592-10623: 1 10624-10655: 5 10656-10687: 7 10688-10719: 327 11872-11903: 2 11904-11935: 1 21088-21119: 1

    Column 2 Total values: 259 Total bytes: 490677 Total oversized values: 0 Total oversized bytes: 0 Total commits: 0 New value insertions: 259 Existing value insertions: 0 Removals: 0 Missed removals: 0 Compressed bytes: 490677 Compression deltas: 0-31: -10 32-63: -6 1024-1055: -36 1056-1087: -160 1088-1119: -190 1120-1151: -330 1152-1183: -220 1184-1215: -200 1216-1247: -100 1248-1279: -20 1280-1311: -30 1472-1503: -33 1504-1535: -11 1568-1599: -48 1600-1631: -48 1632-1663: -168 1664-1695: -96 1696-1727: -204 1728-1759: -96 1760-1791: -72 1792-1823: -65 1824-1855: -91 1856-1887: -26 1888-1919: -26 1920-1951: -13 1952-1983: -39 1984-2015: -26 2016-2047: -39 2080-2111: -56 2112-2143: -70 2144-2175: -42 2176-2207: -28 2208-2239: -28 2240-2271: -14 2272-2303: -14 2304-2335: -15 2400-2431: -15 2432-2463: -15 2464-2495: -15 2496-2527: -15 2560-2591: -16 2592-2623: -48 2624-2655: -16 2688-2719: -16 2848-2879: -17 3840-3871: -21 3872-3903: -21 7040-7071: -33 22528-22559: 895 22720-22751: 943 29440-29471: 1027 29696-29727: 1675 Queries per size tier: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] Missed queries: 0 Value histogram: 0-31: 2 32-63: 1 1024-1055: 4 1056-1087: 16 1088-1119: 19 1120-1151: 33 1152-1183: 22 1184-1215: 20 1216-1247: 10 1248-1279: 2 1280-1311: 3 1472-1503: 3 1504-1535: 1 1568-1599: 4 1600-1631: 4 1632-1663: 14 1664-1695: 8 1696-1727: 17 1728-1759: 8 1760-1791: 6 1792-1823: 5 1824-1855: 7 1856-1887: 2 1888-1919: 2 1920-1951: 1 1952-1983: 3 1984-2015: 2 2016-2047: 3 2080-2111: 4 2112-2143: 5 2144-2175: 3 2176-2207: 2 2208-2239: 2 2240-2271: 1 2272-2303: 1 2304-2335: 1 2400-2431: 1 2432-2463: 1 2464-2495: 1 2496-2527: 1 2560-2591: 1 2592-2623: 3 2624-2655: 1 2688-2719: 1 2848-2879: 1 3840-3871: 1 3872-3903: 1 7040-7071: 1 22528-22559: 1 22720-22751: 1 29440-29471: 1 29696-29727: 1

    Column 3 Total values: 9670092 Total bytes: 406143864 Total oversized values: 0 Total oversized bytes: 0 Total commits: 0 New value insertions: 9670092 Existing value insertions: 0 Removals: 0 Missed removals: 0 Compressed bytes: 406143864 Compression deltas: 32-63: -58020552 Queries per size tier: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] Missed queries: 0 Value histogram: 32-63: 9670092

    Column 4 Total values: 4946090 Total bytes: 1475129728 Total oversized values: 0 Total oversized bytes: 0 Total commits: 0 New value insertions: 4946090 Existing value insertions: 0 Removals: 0 Missed removals: 0 Compressed bytes: 1475129728 Compression deltas: 96-127: 22 256-287: -114561 288-319: -29594022 352-383: 22 544-575: 2153 1120-1151: 73 1920-1951: 468 1952-1983: 166 2080-2111: 171 2272-2303: 192 2528-2559: 217 2848-2879: 250 3072-3103: 274 3360-3391: 304 3680-3711: 339 4000-4031: 374 4032-4063: 764 4064-4095: 382 4352-4383: 412 4384-4415: 848 4768-4799: 455 4800-4831: 1418 5216-5247: 501 5248-5279: 1545 5696-5727: 557 5728-5759: 1711 6208-6239: 612 6816-6847: 678 6848-6879: 2790 7456-7487: 750 7488-7519: 3092 7776-7807: 785 7808-7839: 787 8192-8223: 410512 8224-8255: 11151 8256-8287: 2609 8288-8319: 5218 8320-8351: 4325 8352-8383: 4355 8384-8415: 4388 8448-8479: 4408 8480-8511: 7918 8512-8543: 4450 8544-8575: 4480 8608-8639: 4522 8640-8671: 3609 8672-8703: 3623 8704-8735: 4529 8768-8799: 4590 8800-8831: 4610 8832-8863: 4630 8864-8895: 5814 8928-8959: 4680 8960-8991: 4700 8992-9023: 4720 9024-9055: 4755 9088-9119: 2872 9120-9151: 4785 9152-9183: 4821 9184-9215: 3880 9248-9279: 4880 9280-9311: 7777 9312-9343: 3920 9344-9375: 5910 9408-9439: 4970 9440-9471: 2990 9472-9503: 5013 9504-9535: 5057 9568-9599: 5060 9600-9631: 5085 9632-9663: 5095 9664-9695: 4100 9728-9759: 5134 9760-9791: 4150 9792-9823: 5190 9824-9855: 6256 9888-9919: 4192 9920-9951: 5295 9952-9983: 5295 9984-10015: 5320 10048-10079: 5335 10080-10111: 5370 10112-10143: 5395 10144-10175: 4351 10176-10207: 2096 10208-10239: 5430 10240-10271: 5460 10272-10303: 3284 10304-10335: 6588 10368-10399: 4424 10400-10431: 6640 10432-10463: 2234 10464-10495: 6720 10528-10559: 4481 10560-10591: 4530 10592-10623: 5685 10624-10655: 4552 10656-10687: 2249 10688-10719: 5730 10720-10751: 5730 10752-10783: 5755 10784-10815: 5760 10848-10879: 5840 10880-10911: 5825 10912-10943: 5835 10944-10975: 5885 11008-11039: 5888 11040-11071: 5910 11072-11103: 4748 11104-11135: 7164 11136-11167: 2313 11168-11199: 5990 11200-11231: 4816 11232-11263: 7266 11264-11295: 4849 11328-11359: 4840 11360-11391: 6080 11392-11423: 3642 11424-11455: 7346 11456-11487: 1224 11488-11519: 4936 11520-11551: 6160 11552-11583: 6215 11584-11615: 6240 11648-11679: 6218 11680-11711: 6265 11712-11743: 5027 11744-11775: 6305 11808-11839: 6351 11840-11871: 5085 11872-11903: 6375 11904-11935: 6395 11968-11999: 6457 12000-12031: 6485 12032-12063: 6520 12064-12095: 6540 12128-12159: 6535 12160-12191: 11689 12192-12223: 603618 12224-12255: 10636 12256-12287: 2689 12288-12319: 2720 13376-13407: 2834 14656-14687: 3126 16096-16127: 250271 16128-16159: 3460 16160-16191: 3487 16256-16287: 1744 16320-16351: 1748 16352-16383: 1847 16416-16447: 1756 16480-16511: 1771 16576-16607: 1780 16640-16671: 1788 16736-16767: 1800 16800-16831: 1803 16896-16927: 1815 16960-16991: 3651 17056-17087: 3658 17120-17151: 1832 17216-17247: 1848 17280-17311: 1856 17376-17407: 1867 17440-17471: 1874 17536-17567: 1881 17600-17631: 1890 17696-17727: 1902 17760-17791: 1917 17856-17887: 5770 17920-17951: 1918 18016-18047: 1939 18048-18079: 1943 18080-18111: 1950 18176-18207: 1958 18240-18271: 1971 18336-18367: 1973 18400-18431: 1985 18496-18527: 2000 18560-18591: 4007 18624-18655: 2012 18656-18687: 2018 18720-18751: 2025 18816-18847: 2032 18880-18911: 2037 18976-19007: 2048 19040-19071: 4119 19136-19167: 4141 19200-19231: 2079 19296-19327: 2090 19360-19391: 2103 19456-19487: 4218 19520-19551: 2119 19616-19647: 2123 19680-19711: 2129 19776-19807: 2141 19840-19871: 2154 19936-19967: 2163 20000-20031: 4352 20096-20127: 2178 20160-20191: 2184 20256-20287: 4387 20320-20351: 2197 20416-20447: 2209 20480-20511: 2213 20576-20607: 4469 20608-20639: 2227 20640-20671: 2247 20736-20767: 4505 20800-20831: 4532 20896-20927: 2270 20960-20991: 2280 21056-21087: 2292 21120-21151: 2298 21216-21247: 2309 21280-21311: 2312 21376-21407: 2340 21440-21471: 2342 21536-21567: 2351 21600-21631: 2358 21696-21727: 2364 21760-21791: 2370 21856-21887: 2383 21920-21951: 2400 22016-22047: 2408 22112-22143: 2410 22176-22207: 2423 22240-22271: 2427 22272-22303: 2430 22336-22367: 4868 22400-22431: 2446 22496-22527: 2455 22560-22591: 2458 22656-22687: 4952 22720-22751: 2476 22816-22847: 2493 22880-22911: 2501 22976-23007: 5013 23040-23071: 2514 23136-23167: 5052 23200-23231: 2521 23296-23327: 2542 23360-23391: 2549 23392-23423: 2543 23456-23487: 2547 23520-23551: 2557 23616-23647: 2575 23680-23711: 2592 23776-23807: 2603 23840-23871: 2612 23936-23967: 2608 24000-24031: 2626 24096-24127: 441218 24128-24159: 18543 24160-24191: 2682 Queries per size tier: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] Missed queries: 0 Value histogram: 96-127: 1 256-287: 16376 288-319: 4927632 352-383: 1 544-575: 136 1120-1151: 1 1920-1951: 3 1952-1983: 1 2080-2111: 1 2272-2303: 1 2528-2559: 1 2848-2879: 1 3072-3103: 1 3360-3391: 1 3680-3711: 1 4000-4031: 1 4032-4063: 2 4064-4095: 1 4352-4383: 1 4384-4415: 2 4768-4799: 1 4800-4831: 3 5216-5247: 1 5248-5279: 3 5696-5727: 1 5728-5759: 3 6208-6239: 1 6816-6847: 1 6848-6879: 4 7456-7487: 1 7488-7519: 4 7776-7807: 1 7808-7839: 1 8192-8223: 480 8224-8255: 13 8256-8287: 3 8288-8319: 6 8320-8351: 5 8352-8383: 5 8384-8415: 5 8448-8479: 5 8480-8511: 9 8512-8543: 5 8544-8575: 5 8608-8639: 5 8640-8671: 4 8672-8703: 4 8704-8735: 5 8768-8799: 5 8800-8831: 5 8832-8863: 5 8864-8895: 6 8928-8959: 5 8960-8991: 5 8992-9023: 5 9024-9055: 5 9088-9119: 3 9120-9151: 5 9152-9183: 5 9184-9215: 4 9248-9279: 5 9280-9311: 8 9312-9343: 4 9344-9375: 6 9408-9439: 5 9440-9471: 3 9472-9503: 5 9504-9535: 5 9568-9599: 5 9600-9631: 5 9632-9663: 5 9664-9695: 4 9728-9759: 5 9760-9791: 4 9792-9823: 5 9824-9855: 6 9888-9919: 4 9920-9951: 5 9952-9983: 5 9984-10015: 5 10048-10079: 5 10080-10111: 5 10112-10143: 5 10144-10175: 4 10176-10207: 2 10208-10239: 5 10240-10271: 5 10272-10303: 3 10304-10335: 6 10368-10399: 4 10400-10431: 6 10432-10463: 2 10464-10495: 6 10528-10559: 4 10560-10591: 4 10592-10623: 5 10624-10655: 4 10656-10687: 2 10688-10719: 5 10720-10751: 5 10752-10783: 5 10784-10815: 5 10848-10879: 5 10880-10911: 5 10912-10943: 5 10944-10975: 5 11008-11039: 5 11040-11071: 5 11072-11103: 4 11104-11135: 6 11136-11167: 2 11168-11199: 5 11200-11231: 4 11232-11263: 6 11264-11295: 4 11328-11359: 4 11360-11391: 5 11392-11423: 3 11424-11455: 6 11456-11487: 1 11488-11519: 4 11520-11551: 5 11552-11583: 5 11584-11615: 5 11648-11679: 5 11680-11711: 5 11712-11743: 4 11744-11775: 5 11808-11839: 5 11840-11871: 4 11872-11903: 5 11904-11935: 5 11968-11999: 5 12000-12031: 5 12032-12063: 5 12064-12095: 5 12128-12159: 5 12160-12191: 9 12192-12223: 455 12224-12255: 8 12256-12287: 2 12288-12319: 2 13376-13407: 2 14656-14687: 2 16096-16127: 145 16128-16159: 2 16160-16191: 2 16256-16287: 1 16320-16351: 1 16352-16383: 1 16416-16447: 1 16480-16511: 1 16576-16607: 1 16640-16671: 1 16736-16767: 1 16800-16831: 1 16896-16927: 1 16960-16991: 2 17056-17087: 2 17120-17151: 1 17216-17247: 1 17280-17311: 1 17376-17407: 1 17440-17471: 1 17536-17567: 1 17600-17631: 1 17696-17727: 1 17760-17791: 1 17856-17887: 3 17920-17951: 1 18016-18047: 1 18048-18079: 1 18080-18111: 1 18176-18207: 1 18240-18271: 1 18336-18367: 1 18400-18431: 1 18496-18527: 1 18560-18591: 2 18624-18655: 1 18656-18687: 1 18720-18751: 1 18816-18847: 1 18880-18911: 1 18976-19007: 1 19040-19071: 2 19136-19167: 2 19200-19231: 1 19296-19327: 1 19360-19391: 1 19456-19487: 2 19520-19551: 1 19616-19647: 1 19680-19711: 1 19776-19807: 1 19840-19871: 1 19936-19967: 1 20000-20031: 2 20096-20127: 1 20160-20191: 1 20256-20287: 2 20320-20351: 1 20416-20447: 1 20480-20511: 1 20576-20607: 2 20608-20639: 1 20640-20671: 1 20736-20767: 2 20800-20831: 2 20896-20927: 1 20960-20991: 1 21056-21087: 1 21120-21151: 1 21216-21247: 1 21280-21311: 1 21376-21407: 1 21440-21471: 1 21536-21567: 1 21600-21631: 1 21696-21727: 1 21760-21791: 1 21856-21887: 1 21920-21951: 1 22016-22047: 1 22112-22143: 1 22176-22207: 1 22240-22271: 1 22272-22303: 1 22336-22367: 2 22400-22431: 1 22496-22527: 1 22560-22591: 1 22656-22687: 2 22720-22751: 1 22816-22847: 1 22880-22911: 1 22976-23007: 2 23040-23071: 1 23136-23167: 2 23200-23231: 1 23296-23327: 1 23360-23391: 1 23392-23423: 1 23456-23487: 1 23520-23551: 1 23616-23647: 1 23680-23711: 1 23776-23807: 1 23840-23871: 1 23936-23967: 1 24000-24031: 1 24096-24127: 167 24128-24159: 7 24160-24191: 1

    Column 5 Total values: 4946088 Total bytes: 470504169 Total oversized values: 183 Total oversized bytes: 55944042 Total commits: 0 New value insertions: 4946088 Existing value insertions: 0 Removals: 0 Missed removals: 0 Compressed bytes: 470504169 Compression deltas: 0-31: -19694682 96-127: -380088 128-159: -2478356 160-191: -1529793 192-223: -73406 224-255: -40587 256-287: -213435 288-319: -103936 320-351: -24249 352-383: -10084 384-415: -22685 416-447: 80596 448-479: 471694 480-511: 25936 512-543: 9700 544-575: 30126 576-607: 272221 608-639: -3652 640-671: -13244 672-703: 59059 704-735: 102764 736-767: 235194 768-799: -8253 800-831: 21894 832-863: 61484 864-895: 267264 896-927: 39058 928-959: 17035 960-991: 27311 992-1023: 49251 1024-1055: 119207 1056-1087: 26191 1088-1119: 17407 1120-1151: 26626 1152-1183: 52517 1184-1215: 44541 1216-1247: 22377 1248-1279: 21357 1280-1311: 49071 1312-1343: 52952 1344-1375: 30051 1376-1407: 13148 1408-1439: 19206 1440-1471: 42988 1472-1503: 45141 1504-1535: 15778 1536-1567: 14414 1568-1599: 23241 1600-1631: 42781 1632-1663: 104392 1664-1695: 11789 1696-1727: 48209 1728-1759: 35148 1760-1791: 33033 1792-1823: 27737 1824-1855: 12729 1856-1887: 33449 1888-1919: 27854 1920-1951: 27378 1952-1983: 12953 1984-2015: 18574 2016-2047: 30366 2048-2079: 28750 2080-2111: 26353 2112-2143: 21209 2144-2175: 45837 2176-2207: 28015 2208-2239: 21259 2240-2271: 24115 2272-2303: 16272 2304-2335: 32743 2336-2367: 24702 2368-2399: 27704 2400-2431: 12294 2432-2463: 12599 2464-2495: 39353 2496-2527: 25782 2528-2559: 21141 2560-2591: 71920 2592-2623: 23929 2624-2655: 19701 2656-2687: 23060 2688-2719: 21600 2720-2751: 14150 2752-2783: 28727 2784-2815: 14037 2816-2847: 22089 2848-2879: 3292 2880-2911: 34060 2912-2943: 28103 2944-2975: 18097 2976-3007: 54812 3008-3039: 7745 3040-3071: 37187 3072-3103: 29136 3104-3135: 40329 3136-3167: 8657 3168-3199: 16729 3200-3231: 21665 3232-3263: 19285 3264-3295: 13503 3296-3327: 15189 3328-3359: 25120 3360-3391: 15375 3392-3423: 30117 3424-3455: 25749 3456-3487: 8224 3488-3519: 9435 3520-3551: 13438 3552-3583: 18014 3584-3615: 3272 3616-3647: 11112 3648-3679: 13901 3680-3711: 43210 3712-3743: 31511 3744-3775: 96492 3776-3807: 4575 3808-3839: 22675 3840-3871: 28696 3872-3903: 40642 3904-3935: 5939 3936-3967: 12310 3968-3999: 12068 4000-4031: 15869 4032-4063: 16004 4064-4095: 3506 4096-4127: 3668 4128-4159: 13449 4160-4191: 10432 4192-4223: 4854 4224-4255: 1117 4256-4287: 14532 4288-4319: 9394 4320-4351: 17407 4352-4383: 2747 4384-4415: 2185 4448-4479: 7230 4480-4511: 8341 4512-4543: -51 4544-4575: 2558 4576-4607: 6978 4608-4639: 4343 4704-4735: 1540 4736-4767: 7022 4768-4799: 2640 4800-4831: 790 4832-4863: 8190 4864-4895: 4163 4896-4927: 6993 4960-4991: 12095 4992-5023: 2225 5024-5055: 4740 5120-5151: 736 5152-5183: 3243 5184-5215: 7466 5216-5247: 1228 5280-5311: 3231 5312-5343: 5683 5344-5375: 3403 5376-5407: 889 5408-5439: 2927 5440-5471: 5013 5472-5503: 1273 5504-5535: 4401 5568-5599: 15823 5632-5663: 5315 5664-5695: 3585 5728-5759: 820 5792-5823: 1373 5920-5951: 2806 6016-6047: 2870 6048-6079: 857 6080-6111: 2974 6144-6175: 1695 6208-6239: 3005 6240-6271: 2296 6272-6303: 942 6304-6335: 953 6336-6367: 3863 6368-6399: 5877 6400-6431: 4087 6528-6559: 4195 6560-6591: 1020 6592-6623: 1701 6624-6655: 1976 6656-6687: 3242 6720-6751: 3772 6752-6783: 1017 6784-6815: 2077 6816-6847: 4905 6848-6879: 1083 6880-6911: 4236 6912-6943: 2117 6944-6975: 6811 6976-7007: 3332 7040-7071: 2242 7072-7103: 7670 7104-7135: 6423 7136-7167: 4316 7168-7199: 8962 7200-7231: 4641 7264-7295: 5313 7328-7359: -3 7424-7455: 3759 7488-7519: 85161 7520-7551: 1969 7552-7583: 1858 7584-7615: 1811 7616-7647: 20285 7648-7679: 4055 7680-7711: 6008 7744-7775: 5938 7776-7807: 30384 7808-7839: 3008 7840-7871: 1861 7872-7903: 7398 7904-7935: 4085 7968-7999: 1422 8064-8095: 1748 8128-8159: 2027 8416-8447: 852 8448-8479: 4126 8576-8607: 1568 8896-8927: 2163 9024-9055: 2210 9184-9215: 2306 9344-9375: 2280 9504-9535: 1736 9568-9599: 2305 9632-9663: 2402 9792-9823: 2373 9888-9919: 1940 9920-9951: 1941 10080-10111: 2492 10112-10143: 2001 10144-10175: 1945 10208-10239: 3970 10240-10271: 1996 10272-10303: 1964 10432-10463: 4157 10560-10591: 2049 10624-10655: 5017 10720-10751: 2181 10784-10815: 2155 10816-10847: 2124 10848-10879: 2105 10880-10911: 4308 10976-11007: 2194 11008-11039: 4451 11040-11071: 2214 11072-11103: 2279 11104-11135: 13917 11232-11263: 22693 11264-11295: 2839 11328-11359: 2258 11392-11423: 5455 11648-11679: 5580 11776-11807: 2468 11840-11871: 2940 11904-11935: 2440 12672-12703: 8097 12704-12735: 2686 13184-13215: 3373 13312-13343: 17411 13824-13855: 3539 14048-14079: 3426 14464-14495: 8687 14528-14559: 14626 14592-14623: 3754 14816-14847: 11121 14976-15007: 3845 15072-15103: 3162 15104-15135: 22166 15136-15167: 3910 15392-15423: 7845 15808-15839: 4044 15872-15903: 8206 16032-16063: 3567 16096-16127: 3653 16288-16319: 4155 16384-16415: 7618 16576-16607: 4103 17024-17055: -72 17056-17087: 4375 17408-17439: 4064 17568-17599: -74 17600-17631: -150 17632-17663: -75 17664-17695: -75 17696-17727: -75 17728-17759: -75 17760-17791: -75 17824-17855: -75 17920-17951: -76 17984-18015: -76 18368-18399: 40 18560-18591: 4611 18688-18719: 14589 19168-19199: -53 19200-19231: 4577 19232-19263: 9591 19872-19903: 9297 20512-20543: 4866 22144-22175: 5353 22208-22239: 5280 22272-22303: -93 22368-22399: -93 22464-22495: -94 22528-22559: -94 22560-22591: -94 22592-22623: -282 22624-22655: -94 22656-22687: -282 22688-22719: -283 22720-22751: -380 22752-22783: -284 22784-22815: -380 22816-22847: -848 22848-22879: -475 22880-22911: -190 22944-22975: -95 23104-23135: -95 23360-23391: 5638 23424-23455: 5655 23456-23487: -97 23840-23871: -99 24032-24063: -100 24096-24127: 11605 24224-24255: -101 24256-24287: 5731 24288-24319: -101 25088-25119: 12199 25184-25215: 11846 26016-26047: 6354 26336-26367: 21631 26784-26815: 6470 27648-27679: 6700 28128-28159: 6820 28288-28319: 6712 28608-28639: 7444 28928-28959: 7054 29280-29311: 6986 29408-29439: -121 29440-29471: -242 29472-29503: -484 29504-29535: -363 29536-29567: -242 29600-29631: -366 29632-29663: -244 29664-29695: -244 29696-29727: -244 29760-29791: -122 29792-29823: -244 29824-29855: -366 29856-29887: -369 29888-29919: -246 29920-29951: 32105 29952-29983: -369 29984-30015: -615 30016-30047: -492 30048-30079: -369 30080-30111: -246 30112-30143: -744 30144-30175: -744 30176-30207: -620 30208-30239: -990 30240-30271: -620 30272-30303: -1349 30304-30335: -496 30336-30367: -241 30368-30399: -750 30400-30431: -875 30432-30463: -125 30464-30495: -250 30496-30527: -625 30528-30559: -334 30560-30591: -125 30592-30623: -250 30624-30655: -252 30656-30687: -126 30688-30719: -126 30816-30847: 7568 30848-30879: -99 30880-30911: 7606 31040-31071: 7540 31232-31263: 7569 31328-31359: 8265 31648-31679: 22884 31840-31871: 7634 32288-32319: 8495 32512-32543: 7963 32608-32639: 7920 Queries per size tier: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] Missed queries: 0 Value histogram: 0-31: 3563543 96-127: 63348 128-159: 484488 160-191: 290443 192-223: 37054 224-255: 22190 256-287: 46516 288-319: 204098 320-351: 21168 352-383: 13141 384-415: 17048 416-447: 25977 448-479: 48468 480-511: 9279 512-543: 7224 544-575: 8471 576-607: 17760 608-639: 11501 640-671: 10485 672-703: 4252 704-735: 4489 736-767: 6354 768-799: 6904 800-831: 1674 832-863: 1876 864-895: 3283 896-927: 1888 928-959: 1917 960-991: 871 992-1023: 793 1024-1055: 1248 1056-1087: 669 1088-1119: 624 1120-1151: 387 1152-1183: 497 1184-1215: 426 1216-1247: 310 1248-1279: 233 1280-1311: 264 1312-1343: 298 1344-1375: 178 1376-1407: 132 1408-1439: 110 1440-1471: 177 1472-1503: 174 1504-1535: 91 1536-1567: 55 1568-1599: 83 1600-1631: 124 1632-1663: 197 1664-1695: 48 1696-1727: 79 1728-1759: 90 1760-1791: 80 1792-1823: 65 1824-1855: 28 1856-1887: 55 1888-1919: 61 1920-1951: 64 1952-1983: 25 1984-2015: 32 2016-2047: 64 2048-2079: 58 2080-2111: 58 2112-2143: 28 2144-2175: 53 2176-2207: 55 2208-2239: 35 2240-2271: 37 2272-2303: 21 2304-2335: 43 2336-2367: 39 2368-2399: 45 2400-2431: 14 2432-2463: 19 2464-2495: 51 2496-2527: 32 2528-2559: 36 2560-2591: 59 2592-2623: 32 2624-2655: 28 2656-2687: 30 2688-2719: 23 2720-2751: 14 2752-2783: 34 2784-2815: 17 2816-2847: 29 2848-2879: 5 2880-2911: 36 2912-2943: 33 2944-2975: 23 2976-3007: 43 3008-3039: 8 3040-3071: 39 3072-3103: 29 3104-3135: 31 3136-3167: 7 3168-3199: 19 3200-3231: 26 3232-3263: 17 3264-3295: 15 3296-3327: 13 3328-3359: 25 3360-3391: 16 3392-3423: 23 3424-3455: 18 3456-3487: 7 3488-3519: 7 3520-3551: 12 3552-3583: 18 3584-3615: 3 3616-3647: 10 3648-3679: 13 3680-3711: 32 3712-3743: 20 3744-3775: 170 3776-3807: 6 3808-3839: 16 3840-3871: 24 3872-3903: 70 3904-3935: 5 3936-3967: 13 3968-3999: 9 4000-4031: 16 4032-4063: 24 4064-4095: 5 4096-4127: 4 4128-4159: 10 4160-4191: 14 4192-4223: 3 4224-4255: 2 4256-4287: 7 4288-4319: 9 4320-4351: 8 4352-4383: 2 4384-4415: 3 4448-4479: 7 4480-4511: 4 4512-4543: 3 4544-4575: 1 4576-4607: 7 4608-4639: 4 4704-4735: 2 4736-4767: 6 4768-4799: 2 4800-4831: 1 4832-4863: 15 4864-4895: 2 4896-4927: 6 4960-4991: 4 4992-5023: 3 5024-5055: 4 5120-5151: 1 5152-5183: 1 5184-5215: 6 5216-5247: 1 5280-5311: 1 5312-5343: 6 5344-5375: 3 5376-5407: 1 5408-5439: 2 5440-5471: 2 5472-5503: 1 5504-5535: 2 5568-5599: 5 5632-5663: 4 5664-5695: 1 5728-5759: 1 5792-5823: 2 5920-5951: 2 6016-6047: 2 6048-6079: 1 6080-6111: 2 6144-6175: 2 6208-6239: 2 6240-6271: 2 6272-6303: 2 6304-6335: 1 6336-6367: 1 6368-6399: 2 6400-6431: 1 6528-6559: 3 6560-6591: 1 6592-6623: 2 6624-6655: 2 6656-6687: 3 6720-6751: 3 6752-6783: 1 6784-6815: 2 6816-6847: 4 6848-6879: 1 6880-6911: 4 6912-6943: 4 6944-6975: 3 6976-7007: 3 7040-7071: 2 7072-7103: 7 7104-7135: 4 7136-7167: 4 7168-7199: 10 7200-7231: 7 7264-7295: 3 7328-7359: 1 7424-7455: 2 7488-7519: 23 7520-7551: 1 7552-7583: 1 7584-7615: 1 7616-7647: 5 7648-7679: 1 7680-7711: 2 7744-7775: 3 7776-7807: 8 7808-7839: 1 7840-7871: 1 7872-7903: 2 7904-7935: 1 7968-7999: 1 8064-8095: 1 8128-8159: 1 8416-8447: 1 8448-8479: 2 8576-8607: 1 8896-8927: 1 9024-9055: 1 9184-9215: 1 9344-9375: 1 9504-9535: 1 9568-9599: 1 9632-9663: 1 9792-9823: 1 9888-9919: 1 9920-9951: 1 10080-10111: 1 10112-10143: 1 10144-10175: 1 10208-10239: 2 10240-10271: 1 10272-10303: 1 10432-10463: 2 10560-10591: 1 10624-10655: 2 10720-10751: 1 10784-10815: 1 10816-10847: 1 10848-10879: 1 10880-10911: 2 10976-11007: 1 11008-11039: 2 11040-11071: 1 11072-11103: 1 11104-11135: 4 11232-11263: 3 11264-11295: 1 11328-11359: 1 11392-11423: 2 11648-11679: 1 11776-11807: 1 11840-11871: 1 11904-11935: 1 12672-12703: 1 12704-12735: 1 13184-13215: 1 13312-13343: 2 13824-13855: 1 14048-14079: 1 14464-14495: 1 14528-14559: 3 14592-14623: 1 14816-14847: 2 14976-15007: 1 15072-15103: 1 15104-15135: 2 15136-15167: 1 15392-15423: 2 15808-15839: 1 15872-15903: 2 16032-16063: 1 16096-16127: 1 16288-16319: 1 16384-16415: 1 16576-16607: 2 17024-17055: 1 17056-17087: 1 17408-17439: 1 17568-17599: 1 17600-17631: 2 17632-17663: 1 17664-17695: 1 17696-17727: 1 17728-17759: 1 17760-17791: 1 17824-17855: 1 17920-17951: 1 17984-18015: 1 18368-18399: 1 18560-18591: 1 18688-18719: 1 19168-19199: 1 19200-19231: 1 19232-19263: 1 19872-19903: 1 20512-20543: 1 22144-22175: 1 22208-22239: 1 22272-22303: 1 22368-22399: 1 22464-22495: 1 22528-22559: 1 22560-22591: 1 22592-22623: 3 22624-22655: 1 22656-22687: 3 22688-22719: 3 22720-22751: 4 22752-22783: 3 22784-22815: 4 22816-22847: 9 22848-22879: 5 22880-22911: 2 22944-22975: 1 23104-23135: 1 23360-23391: 1 23424-23455: 1 23456-23487: 1 23840-23871: 1 24032-24063: 1 24096-24127: 1 24224-24255: 1 24256-24287: 1 24288-24319: 1 25088-25119: 2 25184-25215: 1 26016-26047: 1 26336-26367: 1 26784-26815: 1 27648-27679: 1 28128-28159: 1 28288-28319: 1 28608-28639: 1 28928-28959: 1 29280-29311: 1 29408-29439: 1 29440-29471: 2 29472-29503: 4 29504-29535: 3 29536-29567: 2 29600-29631: 3 29632-29663: 2 29664-29695: 2 29696-29727: 2 29760-29791: 1 29792-29823: 2 29824-29855: 3 29856-29887: 3 29888-29919: 2 29920-29951: 5 29952-29983: 3 29984-30015: 5 30016-30047: 4 30048-30079: 3 30080-30111: 2 30112-30143: 6 30144-30175: 6 30176-30207: 5 30208-30239: 8 30240-30271: 5 30272-30303: 11 30304-30335: 4 30336-30367: 2 30368-30399: 6 30400-30431: 7 30432-30463: 1 30464-30495: 2 30496-30527: 5 30528-30559: 3 30560-30591: 1 30592-30623: 2 30624-30655: 2 30656-30687: 1 30688-30719: 1 30816-30847: 1 30848-30879: 1 30880-30911: 1 31040-31071: 1 31232-31263: 1 31328-31359: 1 31648-31679: 3 31840-31871: 1 32288-32319: 1 32512-32543: 1 32608-32639: 1

    Column 6 Total values: 9666 Total bytes: 148113656 Total oversized values: 2 Total oversized bytes: 70722 Total commits: 0 New value insertions: 9666 Existing value insertions: 0 Removals: 0 Missed removals: 0 Compressed bytes: 148113656 Compression deltas: 704-735: 95212 832-863: 1449 1888-1919: 8603 2016-2047: 2305 2272-2303: 4707 2400-2431: 2216 2688-2719: 617 2816-2847: 3889 3072-3103: 3546 3328-3359: 3857 3584-3615: 3332 3744-3775: 864 3872-3903: 895 4000-4031: 3704 4128-4159: 958 4256-4287: 988 4384-4415: 2038 4512-4543: 2101 4640-4671: 1081 4800-4831: 3336 4928-4959: 4572 5184-5215: 3615 5312-5343: 1236 5568-5599: 1300 5696-5727: 3988 5984-6015: 2783 6240-6271: 2907 6368-6399: 1484 6752-6783: 4733 6912-6943: 3216 7040-7071: 4917 7296-7327: 1701 7552-7583: 8818 8096-8127: 1887 8352-8383: 31185 8480-8511: 5940 8608-8639: 10057 8736-8767: 8169 8864-8895: 2073 9152-9183: 34166 9280-9311: 10830 9408-9439: 10987 9536-9567: 11140 9664-9695: 2259 9920-9951: 39491 10080-10111: 9412 10208-10239: 9538 10336-10367: 4832 10464-10495: 7343 10720-10751: 2510 10848-10879: 2540 10976-11007: 48849 11136-11167: 15610 11264-11295: 7898 11392-11423: 5328 11520-11551: 2695 11648-11679: 5452 12032-12063: 53563 12192-12223: 14252 12320-12351: 11522 12448-12479: 5822 12704-12735: 2974 13088-13119: 42937 13248-13279: 30974 13376-13407: 9390 13504-13535: 12641 13632-13663: 3191 13760-13791: 3222 14432-14463: 70929 14560-14591: 27269 14816-14847: 3469 15872-15903: 92943 16000-16031: 7495 16128-16159: 3780 16256-16287: 3812 16544-16575: 3874 17472-17503: 7389479 17600-17631: 2208423 17728-17759: 1058523 17856-17887: 823875 17984-18015: 551909 18112-18143: 577165 18240-18271: 436005 18368-18399: 378915 18528-18559: 338269 18656-18687: 301384 18784-18815: 228742 18912-18943: 203765 19040-19071: 227493 19168-19199: 215589 19296-19327: 176391 19424-19455: 264120 19584-19615: 197144 19712-19743: 184632 19840-19871: 167289 19968-19999: 276040 20096-20127: 169551 20224-20255: 194343 20352-20383: 209972 20480-20511: 211330 20640-20671: 193342 20768-20799: 209179 20896-20927: 220283 21024-21055: 192141 21152-21183: 203271 21280-21311: 204531 21408-21439: 230939 21536-21567: 252562 21696-21727: 223629 21824-21855: 286325 21952-21983: 205739 22080-22111: 227690 22208-22239: 202985 22336-22367: 209453 22464-22495: 194885 22592-22623: 233113 22752-22783: 277108 22880-22911: 251912 23008-23039: 242588 23136-23167: 211450 23264-23295: 272629 23392-23423: 279652 23520-23551: 198544 23648-23679: 327213 23808-23839: 234243 23936-23967: 280387 24064-24095: 140970 24192-24223: 283464 24320-24351: 205253 24448-24479: 349674 24576-24607: 195949 24704-24735: 341883 24864-24895: 320349 24992-25023: 216685 25120-25151: 241362 25248-25279: 242621 25376-25407: 178422 25504-25535: 340830 25632-25663: 228443 25760-25791: 368457 25920-25951: 242911 26048-26079: 347922 26176-26207: 319035 26304-26335: 8780248 26432-26463: 2838387 26560-26591: 1519627 26688-26719: 788694 26720-26751: 62612 26816-26847: 509462 26848-26879: 88063 26976-27007: 568943 27104-27135: 362109 27232-27263: 338337 27360-27391: 301463 27488-27519: 264280 27616-27647: 265562 27744-27775: 6508 28288-28319: 6631 28544-28575: 6694 28672-28703: 6725 28800-28831: 6754 29856-29887: 7005 30400-30431: 7128 30784-30815: 7220 31968-31999: 7500 Queries per size tier: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] Missed queries: 0 Value histogram: 704-735: 634 832-863: 8 1888-1919: 20 2016-2047: 5 2272-2303: 9 2400-2431: 4 2688-2719: 1 2816-2847: 6 3072-3103: 5 3328-3359: 5 3584-3615: 4 3744-3775: 1 3872-3903: 1 4000-4031: 4 4128-4159: 1 4256-4287: 1 4384-4415: 2 4512-4543: 2 4640-4671: 1 4800-4831: 3 4928-4959: 4 5184-5215: 3 5312-5343: 1 5568-5599: 1 5696-5727: 3 5984-6015: 2 6240-6271: 2 6368-6399: 1 6752-6783: 3 6912-6943: 2 7040-7071: 3 7296-7327: 1 7552-7583: 5 8096-8127: 1 8352-8383: 16 8480-8511: 3 8608-8639: 5 8736-8767: 4 8864-8895: 1 9152-9183: 16 9280-9311: 5 9408-9439: 5 9536-9567: 5 9664-9695: 1 9920-9951: 17 10080-10111: 4 10208-10239: 4 10336-10367: 2 10464-10495: 3 10720-10751: 1 10848-10879: 1 10976-11007: 19 11136-11167: 6 11264-11295: 3 11392-11423: 2 11520-11551: 1 11648-11679: 2 12032-12063: 19 12192-12223: 5 12320-12351: 4 12448-12479: 2 12704-12735: 1 13088-13119: 14 13248-13279: 10 13376-13407: 3 13504-13535: 4 13632-13663: 1 13760-13791: 1 14432-14463: 21 14560-14591: 8 14816-14847: 1 15872-15903: 25 16000-16031: 2 16128-16159: 1 16256-16287: 1 16544-16575: 1 17472-17503: 1807 17600-17631: 536 17728-17759: 255 17856-17887: 197 17984-18015: 131 18112-18143: 136 18240-18271: 102 18368-18399: 88 18528-18559: 78 18656-18687: 69 18784-18815: 52 18912-18943: 46 19040-19071: 51 19168-19199: 48 19296-19327: 39 19424-19455: 58 19584-19615: 43 19712-19743: 40 19840-19871: 36 19968-19999: 59 20096-20127: 36 20224-20255: 41 20352-20383: 44 20480-20511: 44 20640-20671: 40 20768-20799: 43 20896-20927: 45 21024-21055: 39 21152-21183: 41 21280-21311: 41 21408-21439: 46 21536-21567: 50 21696-21727: 44 21824-21855: 56 21952-21983: 40 22080-22111: 44 22208-22239: 39 22336-22367: 40 22464-22495: 37 22592-22623: 44 22752-22783: 52 22880-22911: 47 23008-23039: 45 23136-23167: 39 23264-23295: 50 23392-23423: 51 23520-23551: 36 23648-23679: 59 23808-23839: 42 23936-23967: 50 24064-24095: 25 24192-24223: 50 24320-24351: 36 24448-24479: 61 24576-24607: 34 24704-24735: 59 24864-24895: 55 24992-25023: 37 25120-25151: 41 25248-25279: 41 25376-25407: 30 25504-25535: 57 25632-25663: 38 25760-25791: 61 25920-25951: 40 26048-26079: 57 26176-26207: 52 26304-26335: 1424 26432-26463: 458 26560-26591: 244 26688-26719: 126 26720-26751: 10 26816-26847: 81 26848-26879: 14 26976-27007: 90 27104-27135: 57 27232-27263: 53 27360-27391: 47 27488-27519: 41 27616-27647: 41 27744-27775: 1 28288-28319: 1 28544-28575: 1 28672-28703: 1 28800-28831: 1 29856-29887: 1 30400-30431: 1 30784-30815: 1 31968-31999: 1

    Column 7 Total values: 0 Total bytes: 0 Total oversized values: 0 Total oversized bytes: 0 Total commits: 0 New value insertions: 0 Existing value insertions: 0 Removals: 0 Missed removals: 0 Compressed bytes: 0 Compression deltas: Queries per size tier: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] Missed queries: 0 Value histogram:

    Column 8 Total values: 4968297 Total bytes: 1055049526 Total oversized values: 20943 Total oversized bytes: 1387690100 Total commits: 0 New value insertions: 4968297 Existing value insertions: 0 Removals: 0 Missed removals: 0 Compressed bytes: 1055049526 Compression deltas: 0-31: -24730445 288-319: 54 320-351: -6586 640-671: 18755 960-991: 6427 17408-17439: 3194 Queries per size tier: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] Missed queries: 0 Value histogram: 0-31: 4946089 288-319: 1 320-351: 942 640-671: 276 960-991: 45 17408-17439: 1

    Column 9 Total values: 0 Total bytes: 0 Total oversized values: 0 Total oversized bytes: 0 Total commits: 0 New value insertions: 0 Existing value insertions: 0 Removals: 0 Missed removals: 0 Compressed bytes: 0 Compression deltas: Queries per size tier: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] Missed queries: 0 Value histogram:

    Column 10 Total values: 1 Total bytes: 8 Total oversized values: 0 Total oversized bytes: 0 Total commits: 0 New value insertions: 1 Existing value insertions: 0 Removals: 0 Missed removals: 0 Compressed bytes: 8 Compression deltas: 0-31: -5 Queries per size tier: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] Missed queries: 0 Value histogram: 0-31: 1

    Column 11 Total values: 0 Total bytes: 0 Total oversized values: 0 Total oversized bytes: 0 Total commits: 0 New value insertions: 0 Existing value insertions: 0 Removals: 0 Missed removals: 0 Compressed bytes: 0 Compression deltas: Queries per size tier: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] Missed queries: 0 Value histogram:

    Reply
  • v0.2.3 has a broken Cargo.toml
    v0.2.3 has a broken Cargo.toml

    Aug 6, 2021

    I'm trying to follow https://substrate.dev/docs/en/tutorials/create-your-first-substrate-chain/setup

    but my build fails with:

    $ cargo build --release
    error: failed to download `parity-db v0.2.3`
    
    Caused by:
      unable to get packages from source
    
    Caused by:
      failed to parse manifest at `/home/bear/.cargo/registry/src/github.com-1ecc6299db9ec823/parity-db-0.2.3/Cargo.toml`
    
    Caused by:
      failed to parse the version requirement `0.11	` for dependency `parking_lot`
    
    Caused by:
      expected comma after minor version number, found '\t'
    

    I downloaded parity-db==0.2.3 via cargo-download and found that Cargo.toml indeed has an extra \t on the parking_lot declaration:

    # THIS FILE IS AUTOMATICALLY GENERATED BY CARGO
    #
    # When uploading crates to the registry Cargo will automatically
    # "normalize" Cargo.toml files for maximal compatibility
    # with all versions of Cargo and also rewrite `path` dependencies
    # to registry (e.g., crates.io) dependencies
    #
    # If you believe there's an error in this file please file an
    # issue against the rust-lang/cargo repository. If you're
    # editing this file be aware that the upstream Cargo.toml
    # will likely look very different (and much more reasonable)
    
    [package]
    edition = "2018"
    name = "parity-db"
    version = "0.2.3"
    authors = ["Parity Technologies <[email protected]>"]
    description = "Key-value database for the blockchain"
    homepage = "https://substrate.dev"
    license = "MIT OR Apache-2.0"
    repository = "https://github.com/paritytech/parity-db/"
    [profile.release]
    lto = "fat"
    codegen-units = 1
    debug = true
    panic = "abort"
    [dependencies.blake2-rfc]
    version = "0.2.18"
    
    [dependencies.crc32fast]
    version = "1.2.0"
    
    [dependencies.fs2]
    version = "0.4.3"
    
    [dependencies.hex]
    version = "0.4.2"
    
    [dependencies.libc]
    version = "0.2"
    
    [dependencies.log]
    version = "0.4.8"
    
    [dependencies.memmap2]
    version = "0.2"
    
    [dependencies.parking_lot]
    version = "0.11\t"
    
    [dependencies.rand]
    version = "0.8.2"
    [dev-dependencies.env_logger]
    version = "0.8.2"
    
    Reply
  • Remove REF from value table when unneeded.
    Remove REF from value table when unneeded.

    Jul 16, 2021

    This PR remove Rc from value when not needed. When rc is not use, but compression is possible (active in option and on a table above compression threshold), then a bit is added to indicate if compression is used. Also rc uses full 32 byte when no compression is possible.

    Reply
  • Testing from db.
    Testing from db.

    Dec 3, 2021

    This PR extracts some changes I was using in https://github.com/cheme/parity-db/tree/fifo_and_set tests.

    These allow stopping db in a given state to specifically test the commit overlay, log overlay or db file. The tests I added are not very useful for the current code base, but here as an example.

    In https://github.com/paritytech/parity-db/commit/72b1e1d856731fac0f9f2520f2756f2019f183fb I did try to use it on the stress tests, but only work for small number of commit (I did not debug thoroughly as this was a bit expected from the way it is implemented). Not sure it is really useful to have (it did help me, but can be kept outside of master, same thing for the pr actually).

    Reply