Most organizations by now have invested in various software tools and licenses to handle their Big Data. As we move further into agile and more self-service processes, organizations need more capability to store, process, analyze and derive actionable insights from the petabytes of data streaming in from various sources. Dremio promises a new approach to Big Data access and management.
Learn below what Dremio is, its pros and cons, and how Dremio can be leveraged for business intelligence in your organization (and how DashboardFox can help with that 😉).
What Is Dremio?
In the simplest of terms, Dremio is a data lake engine, meaning that you can use Dremio to liberate your data through live and interactive queries sent directly to your cloud-based or on-prem data lake storage.
Since its creation, Dremio has become a popular data lake engine, used to operationalize organizations’ data lake storage and speeding up data science and analytics processes.
Its creators wanted to shatter the traditional norms related to data lake management, and so they created Dremio to remove the barriers that existed in accessing Big Data, as well as bringing control back to users.
The Pros of Dremio
As a high-efficiency and high-performing data lake query engine, Dremio democratizes data access for organizations’ data management teams – data scientists, analysts, and other professionals – through a governed, self-service interface. You enjoy fast and easy analytics at the lowest cost per query for all IT and data professionals in an organization.
Here are some more upsides to using Dremio as your data lake engine:
- High-speed BI dashboarding – Dremio supports business intelligence (BI) dashboarding because of Data Reflections, a feature that invisibly and automatically optimizes parquet data structures and incorporates them into query plan for high-level speeds in queries.
- High-speed data migration – Arrow Flight, another feature in Dremio, easily replaced legacy JDBC and ODBC protocols for up to 1000x faster data migration, suitable for the Big Data workloads in today’s organizations.
- Self-service with IT governance – Dremio puts a semantic layer of data views (called virtual datasets) that enables data professionals to manage, share, and curate data while preserving security and governance – no need to copy data. The layer is fully virtual, searchable, and indexed.
- Joining data from other sources – you can connect/join with external data sources for advanced analytics manipulations without moving the data. This decreases time to value, and you can still migrate data and analytics operations to the data lake storage at your convenience.
The Cons of Dremio
The limitations of Dremio are few, although many are not failings of the platform itself.
Some users have mentioned the following limitations:
- Dremio should offer connections to more legacy data sources – doesn’t affect organizations with newer data sources, only those with legacy systems.
- Dremio should invest in a data dictionary.
- Your use of the platform may be limited by the versions being used – if you have legacy infrastructure, the later versions of Dremio may not be compatible, forcing you to adopt older versions and pass up on the improvements of later versions of Dremio.
- As a relatively new platform (founded in 2015), there is less community information and community support characteristic of open-source platforms.
Dremio offers a free version which is suitable for most small businesses, but if you need more of the security and data governance features, the cost of Dremio can be prohibitive. We’ve heard cost estimates of over $200,000 a year to get into their on-premise enterprise features.
How Dremio Can Be Used For BI
One of the most important aspects of Dremio is that it provides self-service data access – a unified layer that allows any data professional to access the data they need in one place, regardless of where the underlying data is stored.
Data access is a critical facet of any BI operation, but it is increasingly difficult to solve because there are now so many forms of structured and unstructured data and new repositories. Dremio’s approach has been to try providing self-service access to data.
The advent of NoSQL databases set back the journey towards self-service, creating data systems that were more dependent on IT professionals. Dremio looks to reverse this trend by making data self-service possible for data professionals, thereby supporting business intelligence operations.
It accomplished this in the following ways:
- Powered by Apache Arrow –Apache Arrow, designed by Dremio’s creators, is the current standard for columnar in-memory data analytics. Dremio is the first query execution engine that was designed from the outset to take advantage of Apache Arrow. It allows maximum efficiency in query speed and use of memory and computing resources.
- Eliminate latency – low latency is a major read access challenge in cloud data lake storage. Dremio includes Columnar Cloud Cache (C3) that caches and refreshes data in local NVMe storage during access, speeding up NVMe-level data access.
- Data virtualization – instead of making external, post-query data copies (hence creating the challenge of managing and refreshing copies), Dremio uses a virtual view where users can select, combine and transform data into the desired form for access.
These are just a few ways Dremio supports data access for BI, predictive analytics, and all other data science operations in organizations.
Is Dremio the new universal self-service data access system? Not yet, but it does solve many of the challenges of creating unified well-behaved data across an organization. Connected to a suitable business intelligence tool like DashboardFox, organizations will be well on the way to harnessing Big Data for actionable insights well ahead of their competitors.
How Can DashboardFox Help With Dremio
Dremio provides a robust. and scalable option that allows companies of all sizes to bring their data into data lakes. Users can dop new data files, from Excel, CSV, and many other formats into a Dremio folder and set up a process to automatically convert that into a Dremio view that can then be treated as a traditional relational database. Plus Dremio optimized everything for speed.
DashboardFox makes the visualization and presentation of Dremio data available to business users in a self-service and extremely secure fashion.
DashboardFox provides a powerful and affordable, self-hosted BI platform that seamlessly connects to Dremio. DashboardFox is geared toward your internal users and our sister product, Yurbi, which also integrates with Dremio data is for enterprises and software vendors looking for white label embedded analytics.
What makes DashboardFox a better option than Power BI, Tableau, and other Dremio integrated BI tools?
First, we like to point out that no BI platform is best for everyone. So there are going to be use cases where DashboardFox is not a good fit.
Here are some of the things DashboardFox does very well:
Codeless report building - Create ad-hoc reports, datasets, visualizations without needing the assistance of a developer, SQL knowledge, or any proprietary syntax such as DAX from Power BI. DashboardFox is designed for business users and developer skill (nor cost) is needed.
Complete BI Platform - DashboardFox provides the ability to take those reports and convert them into interactive dashboards, mobile views, scheduled email notifications, direct share links, and embeddable dashboards and reports with a variety of personalization and data-level security features (all without requiring development or advanced technical skills).
A Lot of Cost Savings - We don’t like to lead with this, but the reality is DashboardFox. can save you a lot of money. Why? DashboardFox is a one-time fee, not a subscription, like the other BI tools. Pay once, it’s yours to use as you like. We are not charging you monthly or annual to keep using a powerful BI tool to tap into your data.
Contact Us To See For Yourself
We promise you won’t get a hard sales pitch, we’re a team of technical experts that will learn more about what you need and if DashboardFox or Yurbi is not the right fit, we’ll try to point you in the right direction. Schedule a live demo today.