Auto ELT with Rivery's Generative AI Copilot, Blueprint AI

181 zettabytes. That’s the amount of data expected to be created, captured, copied and consumed by 2025, according to Accenture. If raw data is the lifeblood, ELT (extract, load and transform) is the skeleton, giving structure to data for the businesses powering our data consumption needs. Moving data from one place to another, as trivial as it sounds, has always been a laborious process; adjusting for changes in structures, arranging connections to new data sources, and fitting them into existing flows. That process is getting face-lift with Generative AI and copilots, courtesy of Rivery.

Rivery Blueprint is an AI-powered copilot designed to simplify and accelerate data integration at scale. "We're here to make data integration simple and flexible for the changes that happen every week," says Itamar Ben Hemo, CEO of Rivery. "Our vision is to democratize data integration, and this AI-powered copilot is a significant step towards that goal."

Blueprint’s technology can analyze and understand the structure of new data sources at the user’s request, generate YAML configuration files for seamless integration, and enable users to quickly tap into new data sources and integrate them into their data workflows. The copilot analyze API documentation to learn all the different settings, such as pagination, rate limits, and error handling, necessary for proper API calls.

One of the key benefits of Rivery's copilot is its ability to rapidly develop connectors. What once took a month can now be accomplished in a fraction of the time, allowing businesses to integrate new data sources at unprecedented speed and a reduced maintenance burden on data teams.

“Unlike black-box solutions, our copilot generates YAML files that can be reviewed, modified, and version-controlled by data teams,” added Ben Hemo. This transparency ensures that data professionals maintain control over their integrations while benefiting from AI assistance.

While Blueprint alleviates the data integration challenges for organizations, 68% of which expect their overall data volume to increase over the next 12 months according to a Salesforce's report, it also lends itself particularly well to data engineers of Large Language Model (LLM) developers. "We're witnessing an explosion of GenAI apps. The demand for analytics is skyrocketing. Engineers need a faster, more efficient way to unify quality data sources to deliver value for businesses and consumers."

The system incorporates strong text validation, leveraging RAG (Retrieval-Augmented Generation) and AI engines to catch potential issues and suggest improvements, further enhancing the quality and reliability of data integrations.

“We're excited to introduce Rivery Blueprint, our Generative AI solution to the ever changing Rubik's Cube that is ELT. No more poring over API documentation. No more manual pipeline coding. This is the future of data integration."

Rivery Blueprint is available in private preview with a group of customers. It will be released to the general public later this year.