Kingland Systems has made use of traditional relational database systems since our very first security broker application rolled out in the early 1990s. For the most part, these relational database systems have worked well for us, especially when we manage predictable data with simple and consistent relationships. However, increasingly our applications are managing large volumes of interconnected data as well as semi-structured or unstructured data that relational database systems are not so well equipped to deal with. An example is the hierarchy of companies and subsidiaries (entities). 
At Kingland we work with entity hierarchies (parent companies and their subsidiaries) and other inter-related data. We have used different relational database systems over the years to manage this information. Relational database systems model data as a set of tables, rows and columns. This requires complex joins, self-joins and recursive queries when the dataset becomes more inter-related. Such queries are complex to construct and expensive to run. For instance, within entity hierarchies it is not uncommon for a subsidiary to have multiple parent companies, making the tree traversal logic on a relational database very challenging. Making them perform well is difficult, especially on a large dataset. In more recent years, we have started using another type of database, optimized for connected data: the graph database. The graph data model has allowed us to find interconnected data much faster and in a much more scalable manner as compared to the relational data model.
Another common problem we work with is consolidating data from various sources, usually in many different formats, and sometimes not following any particular format at all. We ingest such semi-structured or unstructured data into our systems for information gathering through automated and manual workflows. Often the volume is very large. This is an example of where we have found key/value or document based NoSQL databases to work well, giving us high performance, easy scalability, and rich query capabilities on a dataset that would have been difficult, if not impossible, to manage in a relational database system.
So what are some of the factors you should consider as you take the various database options into the fitting room? One of the first things to look at is what kind of queries you will run. Relational databases have the advantage of powerful and flexible queries across all the data in the database. Graph databases have highly specialized query capabilities that make them the better choice for interconnected data. Key/value and document NoSQL databases have more primitive querying and searching capabilities, but can retrieve data really fast, even on very large datasets. Other things to consider are your needs for ACID compliance (atomicity, consistency, isolation, and durability), security (access control and encryption at rest), fault tolerance (avoiding data loss), and scalability (large data volumes).
The Kingland Platform follows a microservice architecture. Each microservice is a separate executable that can be based on different technologies, allowing us to pick those that are best suited to address the needs being met. As we have built out our platform, we have taken our various database needs to the fitting room, selecting the right engine for the given workload (microservice). In the Enterprise Data Management platform area alone, we have components using relational, graph, nosql, caching, and search engines as their storage mechanism. This provides our platform customers benefits in terms of performance, scalability, maintainability, and concurrency.
If you have been using a particular engine to address all your database needs, I encourage you to try others – you might find some that are a better fit. Or contact us to see how our platform components with their optimized storage mechanisms can help you solve your software needs.
1401 6th Ave South, Clear Lake, IA 50428
1-641-355-1000
Comments (1)