Skip to main content

Command Palette

Search for a command to run...

Day 22/100

Published
•2 min read•View as Markdown

Designing Data-Intensive Applications [Book Highlights]

[Part I : Chapter II]

Data Models and Query Languages

  • The goal of the relational model was to hide that implementation detail behind a cleaner interface.
  • Object-relational mapping (ORM) frameworks like ActiveRecord and Hibernate reduce the amount of boilerplate code required for this translation layer, but they can’t completely hide the differences between the two models.
  • In a relational database, the query optimizer automatically decides which parts of the query to execute in which order, and which indexes to use.
  • Schema flexibility in the document model Most document databases, and the JSON support in relational databases, do not enforce any schema on the data in documents.
  • If the data in your application has a document-like structure (i.e., a tree of one-to many relationships, where typically the entire tree is loaded at once), then it’s probably a good idea to use a document model. The relational technique of shredding— splitting a document-like structure into multiple tables can lead to cumbersome schemas and unnecessarily complicated application code.
  • Schema-on-read is similar to dynamic (runtime) type checking in programming languages, whereas schema-on-write is similar to static (compile-time) type checking.
  • If your application often needs to access the entire document, there is a performance advantage to this storage locality.
  • The locality advantage only applies if you need large parts of the document at the same time.
  • It seems that relational and document databases are becoming more similar over time, and that is a good thing

100 Days of Data Engineering

Part 8 of 20

I have taken up a challenge for myself to spend at least 1 hour everyday for 100 straight days on reading/coding data engineering stuff and will also be summarizing daily learnings under this series.

Up next

Day 23/100

Spark Streaming from Amazon SQS So this is a journal log, to describe how was I able to integrate Spark Streaming and Amazon SQS and what all problems I faced while doing so. Problem Statement Seems like when we read s3 files from spark Streaming, s...