Rich Hickey: Deconstructing the Database

We recently started using Solr in a similar manner where I work. We don't go as far as to pack an index in with our code, but each page serving instance has an embedded Solr server that it uses to pull a lot of fairly static data. Those indexes are updated nightly from a master Solr server which imports its data from a normal relational database. So far it has worked really well, but I don't think it really offers many advantages scalability wise over other NoSQL solutions. Like most other solutions it trades fast writes/consistent reads for fast reads, which is fine in many cases. Around the time we were adding Solr though, I heard about Datomic ( [LINK] ). I haven't had time to really investigate it, but it seems to me that Datomic provides fast reads in a manner very similar to the Solr configuration you described. Live versions of the database's index are deployed to agents that live on your app servers, but unlike the Solr configuration these agents receive updates to the index in real time. At a glance at least, it seems to offer all of the advantages of the Solr solution while still allowing relatively fast updates to the data. This is an interesting talk on the topic by Rich Hickey (who created Datomic): [VIDEO] . It's long but I thought it was worthwhile.

Although tangential to your question, I feel you might enjoy presentations by Rich Hickey wherein he describes traditional database architectures and presents the whys and wherefores behind Datomic. For example: [1] [VIDEO] [2] [LINK]

You will enjoy the fact-based database Datomic, which is the reason Rich Hickey made Clojure: [VIDEO] Datalog is a much, much, much query language better than SQL: [LINK]

Rich Hickey's talk on the design of Datomic is also something every programmer should see at least once: [VIDEO]