CppCon 2014: Chandler Carruth "Efficiency with Algorithms, Performance with Data Structures"

There are many things in the standard C++ library which are named incorrectly (std::string), awkwardly (std::unordered_map) and and implemented inefficiently from the perspective of modern CPUs (same std::unordered_map, which uses linked list for the underlying hash table buckets). See the great talk on CppCon 2014 about those issues - [VIDEO] .

Chandler Carruth's CPPCon talk [0] this year was also great (surprisingly, as I had a fairly low opinion of most of the others). It wasn't better than Mike Acton's [1], but I suspect that it was more actionable for most devs. [0] [VIDEO] [1] [LINK]

Also, those allocations imply linked, rather than contiguous, data structures, which (often? always?) imply poor cache interaction, which is a major source of inefficiency. Would love some more perspectives on this, as I've just watched a couple cppcon talks ([0], [1]) and I'm feeling a bit drunk on the kool-aid. [0]: [VIDEO] [1]: [LINK]

I'm pretty curious about this: have memory access patterns and cache awareness become more of a big deal in the past decade or so, or has it only recently reached my awareness? I studied CS in the early 2000s and while I was aware of various trade-offs between contiguous and linked data structures, it's only in the last few years that the importance of cache (and non-"system" languages' widespread flouting of its importance) has really bubbled up in my consciousness. It's certainly likely that I just didn't take the right kinds of classes, or that I did but just missed or failed to really internalize this, but I'm curious if something else is going on. Has the bottleneck increasingly become memory rather than CPU? Have we more thoroughly exhausted other areas of optimization? I was thinking about this while watching a talk[0] given by someone from Google about data structures in the STL, which decried that the standardized specification for `std::unordered_map` more or less forces an implementation to use buckets instead of the more cache-friendly technique of probing. I'm sure that the people who standardized it were very competent, which makes me think that, at least at the time, buckets were thought to be the better implementation. [0]: [VIDEO] Edit: Added video link.

Here's a few: - "Tuning C++: Benchmarks, and CPUs, and Compilers! Oh My!" [LINK] - "Understanding Compiler Optimization" [LINK] - "Optimizing the Emergent Structures of C++" [LINK] - "Efficiency with Algorithms" [VIDEO] The first one is really good if you've never benchmarked anything in a Linux environment. Or if you otherwise want to learn how to investigate how C++ gets turned into machine code.

I think C++'s built in unordered_map has to use chaining due to restrictions on iterators mandated for standard library containers, which is a drag. You've still got to roll your own open addressed container. :( This comes up in one of Chandler Carruth's talks [1]. [1] [VIDEO]

STL is not inherently slow per se, some data structures are just not fast on modern CPUs, so they have to be chosen wisely. Here is good talk: [VIDEO] .

Uniformly shitty code is hard to profile. See for example what Chandler has to say about writing good code [VIDEO]

Chandler Carruth talks about unordered_map performance[1] in his CppCon 2014 talk "Efficiency with Algorithms, Performance with Data Structures". He endorses most of the other alternatives you mentioned. [1] [VIDEO]

Yes, if you write performance sensitive code you have to be very careful with dictionaries. See for example this nice talk by Chandler Carruth [VIDEO]

Somewhat interestingly, C++ STL unordered_map is so much slower than it needs to be: [VIDEO] Using one of the policy-based data structures instantly gets you about an order magnitude of speedup (for accesses).

This talk seems to me to follow a similar line of thinking to the one I saw presented by Chandler Carruth at the 2014 C++ conference [0]. In the talk he presented a table with (approximately) round-trip-times of various data layers. [0]: [VIDEO]

If you haven't already seen it, you'd probably really enjoy Chandler Carruth's "Efficiency with Algorithms, Performance with Data Structures" talk from CppCon 2014: [VIDEO]

But then your objects are scattered all over the address space, which means that objects you’re accessing in sequence are unlikely to be in the same cache line, so you pay a cache miss on every single access. Having (maybe?) more efficient malloc/free in exchange for dramatically increasing the cost of accessing your data is not a good trade-off for most access patterns. See Chandler Carruth’s talk from CppCon 2014: [VIDEO]

Strongly agree. That's perhaps less true for the software I work on these days (lapack), but I've seen that so many times over my career. I'm also a big fan of "Efficiency with Algorithms, Performance with Data Structures" by Chandler Carruth at CppCon 2014. [VIDEO]