Succinct Data Structures

I find it mildly amusing that people are competing on the fastest decoder of horribly inefficient format. Don't get me wrong, I like e.g. ZX Spectrum demos, but the practical utility of such endeavor is questionable. But that aside, I wonder how this compares to the compressed indexes approaches (see [VIDEO] and [LINK] ), obviously, once the index is built.

I don't feel like it tonight. Drinking whiskey and trying not to think about how effed up the Israel footage and the North Korea situation are. Maybe that's why it's not the best post. I don't mean to crap on the work presented. Great article, good summary, and the tech is solid. It's just older than what excites me; a lot of progress has been made in 20 years and the majority of it hasn't found commercial applications. But here is a citation root for a lot of amazing work in this space: [LINK] My absolute hero Edward Kmett gave a talk stitching a lot of this work together a long time ago: [VIDEO] . I have no idea if he's pursued it, it's just one of his many talks that left me with an incredibly lasting impression. Variants of this technique work for arbitrary documents and structures, work better at very high volume, have cache oblivious properties, and support transactions. Universal indexes that are reasonably good for all queries (as opposed to specfiic queries) are also possible. Coupled with discrimination-based techniques for O(n) table joins, there's probably a whole startup around there. Sorry I can't do better right now.

I first heard of the concept of succinct data structures from Edward Kmett, a famous Haskeller behind many popular Haskell libraries. He gave a talk on succinct data structures a long time ago: [VIDEO]