As machine learning is being adopted in every business, it’s become clear that the next major challenge is in how we enable teams and businesses to make good use of ML models. While data scientists are trained to build useful machine learning models, the (engineering) skills required to integrate those models into the business are completely different. MLOps tools set out to solve this problem, but they have led us down the wrong path by forcing data teams to grapple with low-level cloud infrastructure to accomplish everyday tasks.
We’ve talked to 175+ data teams to better understand their challenges today. Based on our conversations, we believe the missing link is a solution for production data science (PDS), not MLOps.1 Production data science infrastructure takes the opposite approach from MLOps: Rather than exposing and expanding the complexity of low-level cloud infrastructure, PDS infrastructure manages the underlying infrastructure while enabling data teams to easily deploy models anywhere, publish predictions consistently, and ensure ongoing model quality.
At Aqueduct, we’re building an open-source production data science platform designed and built for data teams to help make data science projects useful quickly.
Production data science (PDS) infrastructure enables data scientists to repeatably deliver high-quality predictions to their business without having to manage low-level cloud infrastructure tools. At its core, PDS covers 3 critical tasks:
Until recently, no existing tools met these requirements. That’s why we built Aqueduct.
The Aqueduct open-source project enables data scientists to go from insight to impact by automating the engineering needed to connect models to the data, services, and people that need them. Aqueduct enables turnkey productionization of data science projects — whether it’s a simple heuristic-based workflow running locally or large prediction task running in the cloud — and is purpose-built to meet the three core needs of production data science:
We’re really excited about Aqueduct. If what we’re building is interesting or useful for you, we’d love to hear from you! Check out what we’re building, join our Slack community, and let us know what you think!
1If you're interested in learning more about how Production Data Science is different from MLOps, check out the philosophy behind Aqueduct for more detail.