Enterprise Data Storage on

Exploring GraphDB and Neo4j - A Guide to Graph Databases

hari@dasarpai.com (Dr. Hari Thapliyaal) — Mon, 10 Mar 2025 00:00:00 +0000

RDBMS vs. Neo4j (Cypher) Command Comparison
#

Are GraphDB faster than RDBMS?
#

Graph databases (GraphDBs) can be faster than relational databases (RDBMS) in scenarios that involve complex relationships and deep traversals, but they are not always universally faster. It depends on the query type, data structure, and use case.

Selecting Database for Project

hari@dasarpai.com (Dr. Hari Thapliyaal) — Sat, 05 Oct 2024 00:00:00 +0000

Essential Database Selection Criteria for Modern Applications
#

Is this article for me?
#

If you are looking answer for these questions then “Yes”.

What parameters should you consider to choose a database for your project?
What are different data formats for bigdata?
What is the difference between OCR and Parquet data formats?
What is CAP Theorem?
What is the difference between Sharding and Partitioning

What parameters should you consider to choose a database for your project?
#

When you are developing an application which need to store the data or you need to pull data from some format for your project work that time you need to take care of many parameters. Sometimes it looks there is an ovbious choice to go for a certain type of database for some specific work but most of the time it is challenging. What are those aspects of a database which you need to take care of?

Exploring Apache Hive

hari@dasarpai.com (Dr. Hari Thapliyaal) — Fri, 04 Oct 2024 00:00:00 +0000

Exploring Apache Hive: Capabilities and Scalability for Big Data Processing
#

What is Hive?
#

Apache Hive is a data warehousing and SQL-like query engine built on top of Hadoop. It provides a platform for processing large datasets stored in Hadoop Distributed File System (HDFS) and other data storage systems that integrate with Hadoop. Hive simplifies querying and managing big data with a familiar SQL-like syntax (HiveQL). Below are the key capabilities of Hive:

Serverless databases

hari@dasarpai.com (Dr. Hari Thapliyaal) — Tue, 23 Jul 2024 00:00:00 +0000

Serverless databases
#

A serverless database is a type of database service that automatically manages infrastructure and scaling, allowing developers to focus solely on building applications without having to worry about server provisioning, capacity planning, or database maintenance.

Key Characteristics of Serverless Databases:
#

Automatic Scaling:

SQL and Relational Algebra

hari@dasarpai.com (Dr. Hari Thapliyaal) — Mon, 14 Aug 2023 00:00:00 +0000

SQL and Relational Algebra
#

Relational algebra (RA) is considered as a procedural query language where the user tells the system to carry out a set of operations to obtain the desired results. i.e. The user tells what data should be retrieved from the database and how to retrieve it.

Relational algebra notions can be implemtned via any any SQL language like PL/SQL, TSQL, SQLite SQL, DB2 SQL, MariaDB SQL, FireBird SQL, PSQL, ANSI SQL commands in any databases like MySQL, PostgreSQL, Oracle, SQLServer SQL server.

Database and Analytics Product Services from Google Azure AWS

hari@dasarpai.com (Dr. Hari Thapliyaal) — Fri, 21 Jul 2023 00:00:00 +0000

Database and Analytics Product Services from Google Azure AWS
#

Database Tools/Services
#

Sno	Amazon	Azure	Microsoft
1.	Amazon Aurora : High performance managed relational database with full MySQL and PostgreSQL compatibility	SQL : Managed MySQL, PostgreSQL, SQL Server	Azure SQL : Migrate, modernize, and innovate on the modern SQL family of cloud databases
2.	Amazon Aurora Serverless V2 : Instantly scale to >100,000 transactions per second	Datastore : NoSQL database for your web and mobile apps	Azure Cosmos DB : Build or modernize scalable, high-performance apps
3.	Amazon DocumentDB (with MongoDB compatibility) : Fully managed document database	Firestore : Serverless NoSQL document DB	Azure SQL Database : Build apps that scale with managed and intelligent SQL database in the cloud
4.	Amazon DynamoDB : Managed NoSQL database	Spanner : Horizontally scalable relational DB	Azure Database for PostgreSQL : Fully managed, intelligent, and scalable PostgreSQL
5.	Amazon ElastiCache : In-memory caching service	Bigtable : Petabyte-scale, low-latency, non-relational	Azure SQL Managed Instance : Modernize SQL Server applications with a managed, always-up-to-date SQL instance in the cloud
6.	Amazon Keyspaces (for Apache Cassandra) : Managed Cassandra-compatible database	Memorystore : Managed Redis and Memcached	Azure Database for MySQL : Fully managed, scalable MySQL Database
7.	Amazon MemoryDB for Redis : Redis-compatible, durable, in-memory database that delivers ultra-fast performance	Database Migration : Cloud SQL migrations simplified	SQL Server on Azure Virtual Machines : Migrate SQL Server workloads to the cloud at lower total cost of ownership (TCO)
8.	Amazon Neptune : Fully managed graph database service	MongoDB Atlas : JSON-like data models, querying, & scaling	Azure Cache for Redis : Accelerate apps with high-throughput, low-latency data caching
9.	Amazon Quantum Ledger Database (QLDB) : Fully managed ledger database	Neo4j Aura : Integrated, fully managed graph databases	Azure Database Migration Service : Accelerate your data migration to Azure
10.	Amazon RDS : Managed relational database service for MySQL, PostgreSQL, Oracle, SQL Server, and MariaDB	Redis Enterprise : Robust in-memory database platform	Azure Managed Instance for Apache Cassandra : Modernize Cassandra data clusters with a managed instance in the cloud
11.	Amazon RDS on Outposts : Automate on-premises database management	AlloyDB : Enterprise-grade, PostgreSQL-compatible databases	Azure Database for MariaDB : Deploy applications to the cloud with enterprise-ready, fully managed community MariaDB
12.	Amazon Redshift : Fast, simple, cost-effective data warehousing
13.	Amazon Timestream : Fully managed time series database
14.	AWS Database Migration Service : Migrate databases with minimal downtime

Data Analytics Tools/Services
#

Sno	Amazon	Azure	Microsoft
1.	Amazon Athena : Query data in S3 using SQL	BigQuery : Data warehouse for business agility and insights.	Azure Synapse Analytics : Limitless analytics with unmatched time to insight
2.	Amazon Cloud : SearchManaged search service	Looker : Platform for BI, data applications, and embedded analytics.	Azure Databricks : Design AI with Apache Spark™-based analytics
3.	Amazon DataZone (Preview) : Unlock data across organizational boundaries with built-in governance	Dataflow : Streaming analytics for stream and batch processing.	Microsoft Purview : Govern, protect, and manage your data estate
4.	Amazon OpenSearch Service : Search, visualize, and analyze up to petabytes of text and unstructured data	Pub/Sub : Messaging service for event ingestion and delivery.	Azure Data Factory : Hybrid data integration at enterprise scale, made easy
5.	Amazon EMR : Easily run big data frameworks	Dataproc : Service for running Apache Spark and Apache Hadoop clusters.	HDInsight : Provision cloud Hadoop, Spark, R Server, HBase, and Storm clusters
6.	Amazon FinSpace : Analytics for the financial services industry	Cloud Data Fusion : Data integration for building and managing data pipelines.	Azure Stream Analytics : Real-time analytics on fast-moving streaming data
7.	Amazon Kinesis : Analyze real-time video and data streams	Cloud Composer : Workflow orchestration service built on Apache Airflow.	Azure Machine Learning : Build, train, and deploy models from the cloud to the edge
8.	Amazon Managed Streaming for Apache Kafka : Fully managed Apache Kafka service	Dataprep : Service to prepare data for analysis and machine learning.	Azure Analysis Services : Enterprise-grade analytics engine as a service
9.	Amazon Redshift : Fast, simple, cost-effective data warehousing	Dataplex : Intelligent data fabric for unifying data management across silos.	Azure Data Lake Storage : Scalable, secure data lake for high-performance analytics
10.	Amazon QuickSight : Fast business analytics service	Looker Studio : Interactive data suite for dashboarding, reporting, and analytics.	Azure Data Explorer : Fast and highly scalable data exploration service
11.	AWS Clean Rooms : Match, analyze, and collaborate on datasets–without sharing or revealing underlying data	Analytics Hub : Service for securely and efficiently exchanging data analytics assets.
12.	AWS Data Exchange : Find, subscribe to, and use third-party data in the cloud
13.	AWS Data Pipeline : Orchestration service for periodic, data-driven workflows
14.	AWS Glue : Simple, scalable, and serverless data integration
15.	AWS Lake Formation : Build, manage, and secure your data lake

Demystifying DevOps, MLOps, and DataOps

hari@dasarpai.com (Dr. Hari Thapliyaal) — Thu, 08 Jun 2023 00:00:00 +0000

Demystifying DevOps, MLOps, and DataOps:
#

Bridging the Gap between Software Development, Machine Learning, and Data Managemen

Introduction
#

What is DevOps
#

DevOps, short for Development and Operations, is a set of practices, principles, and cultural philosophies that aim to improve collaboration and efficiency between software development teams and IT operations teams. It emphasizes the integration of software development and IT operations, breaking down traditional silos and fostering a collaborative approach throughout the entire software delivery lifecycle.

Data Lake vs Data Warehouse vs Data Mart vs Lakehouse

hari@dasarpai.com (Dr. Hari Thapliyaal) — Wed, 25 Jan 2023 08:33:00 +0530

What is the difference between Data Lake, Data Warehouse, Data Mart, and Lakehouse?
#

Introduction
#

Enterprise data platforms have moved well beyond the simple lake-versus-warehouse debate. Today you will also hear about data lakehouses, data mesh, data fabric, and layered designs such as the medallion architecture. From a database perspective, the core question is still the same: where does data live, how is it structured, and who can query it reliably?

Type of Databases

hari@dasarpai.com (Dr. Hari Thapliyaal) — Wed, 25 Jan 2023 08:33:00 +0530

What are the various types of databases?
#

Introduction
#

In the 21st Century, Data is the real oil of machines. There are different kinds of oils and there are different kinds of containers. Similarly, there are different kinds of data and there are different kinds of database software to manage these databases. Broadly we call them SQL and NoSQL databases. But In the NoSQL, there are some other finer groups for specific purposes. For how those database types fit into lakes, warehouses, lakehouses, and marts at enterprise scale, see Data Lake vs Data Warehouse vs Data Mart vs Lakehouse. Let’s look into these.

Navigating the Data Landscape: Exploring Data Sources, Databases, and ETL Tools for Machine Learning Projects

hari@dasarpai.com (Dr. Hari Thapliyaal) — Fri, 16 Jul 2021 00:00:00 +0000

Navigating the Data Landscape:
#

Exploring Data Sources, Databases, and ETL Tools for Machine Learning Projects

Introduction
#

Data sources: Data sources refer to the origins or locations from which data is collected or generated. They can include various platforms, systems, devices, or applications that generate or store data, such as databases, APIs, files, sensors, social media platforms, or web services.

Enterprise Data Storage on

Exploring GraphDB and Neo4j - A Guide to Graph Databases

RDBMS vs. Neo4j (Cypher) Command Comparison #

Are GraphDB faster than RDBMS? #

Selecting Database for Project

Essential Database Selection Criteria for Modern Applications #

Is this article for me? #

What parameters should you consider to choose a database for your project? #

Exploring Apache Hive

Exploring Apache Hive: Capabilities and Scalability for Big Data Processing #

What is Hive? #

Serverless databases

Serverless databases #

Key Characteristics of Serverless Databases: #

SQL and Relational Algebra

SQL and Relational Algebra #

Database and Analytics Product Services from Google Azure AWS

Database and Analytics Product Services from Google Azure AWS #

Database Tools/Services #

Data Analytics Tools/Services #

Related reading #

Demystifying DevOps, MLOps, and DataOps

Demystifying DevOps, MLOps, and DataOps: #

Introduction #

What is DevOps #

Data Lake vs Data Warehouse vs Data Mart vs Lakehouse

What is the difference between Data Lake, Data Warehouse, Data Mart, and Lakehouse? #

Introduction #

Type of Databases

What are the various types of databases? #

Introduction #

Navigating the Data Landscape: Exploring Data Sources, Databases, and ETL Tools for Machine Learning Projects

Navigating the Data Landscape: #

Introduction #

RDBMS vs. Neo4j (Cypher) Command Comparison
#

Are GraphDB faster than RDBMS?
#

Essential Database Selection Criteria for Modern Applications
#

Is this article for me?
#

What parameters should you consider to choose a database for your project?
#

Exploring Apache Hive: Capabilities and Scalability for Big Data Processing
#

What is Hive?
#

Serverless databases
#

Key Characteristics of Serverless Databases:
#

SQL and Relational Algebra
#

Database and Analytics Product Services from Google Azure AWS
#

Database Tools/Services
#

Data Analytics Tools/Services
#

Related reading
#

Demystifying DevOps, MLOps, and DataOps:
#

Introduction
#

What is DevOps
#

What is the difference between Data Lake, Data Warehouse, Data Mart, and Lakehouse?
#

Introduction
#

What are the various types of databases?
#

Introduction
#

Navigating the Data Landscape:
#

Introduction
#