Reflection on the Data AI Summit 2024: Insights from our CGO and CTO

In this blog, our CGO, Sam Verdonck, and our CTO, Simon Mestdagh, share their insights from the DataAISummit 2024. Read on to discover how the latest developments in data and AI can transform your organization.

‍

Last month, we announced the integration of Tangent as a Databricks runtime. For more details, check out our blog article, "Elevating Time Series AI with Tangent Works and Databricks" written by our CTO, Simon Mestdagh.

‍

Breaking Down Data Silos with Databricks and the Open Sourcing of Unity Catalog

Sam: "The announcement from Databricks to open source their Unity Catalog is a significant step forward for organizations looking to centralize, unify, and structure their data. This move is especially important as it addresses the long-standing issue of data silos. By providing a unified governance solution across various data formats and platforms, Databricks is enabling companies to prepare their data more effectively for automated and real-time predictive AI applications. This is critical given that less than 15% of AI models are operational today."

‍

Simon: "Absolutely, Sam. Open sourcing Unity Catalog under an Apache 2.0 license means more flexibility and interoperability across different data systems and AI tools. This will not only help in reducing costs but also improve the scalability of AI solutions.  

‍

The Importance of Custom Data Models in Enterprise AI

Simon: "While foundational models trained on public data are valuable, enterprises often require models tailored to their specific datasets and business needs. This is where tools like Tangent come into play. Tangent allows for the efficient and cost-effective training of models on custom data, which is essential for achieving high performance and agility in AI operations."

‍

Sam: "That’s a great point, Simon. The ability to leverage custom data ensures that the AI models are more relevant and accurate for specific business contexts. It also facilitates a more personalized and impactful use of AI across various business functions, driving better outcomes and more strategic insights."

‍

‍‍

Advancing Data Quality and Security

Sam: "Data quality and security remain at the forefront of enterprise concerns, but there is significant progress being made. Leading companies are now focusing on these foundational aspects, which is enabling them to scale their AI and MLOps practices more effectively."

‍

Simon: "Indeed, ensuring data quality and robust security protocols is critical for building trust in AI systems. With advancements in data governance tools and methodologies, organizations can now manage their data more reliably, paving the way for broader AI adoption and innovation."

‍

Alternatives to Feature Stores and MLFlow for Time Series MLOps

Sam: "Many enterprises are prioritizing the creation of feature stores and implementing tools like MLFlow to support their MLOps. However, it’s interesting to note that for Time Series MLOps, there are viable alternatives that don’t necessarily require these tools."

‍

Simon: "Time Series MLOps can indeed be managed with other specialized tools and methodologies that may offer more streamlined or cost-effective solutions depending on the specific use case and infrastructure of the organization."

‍

‍

Overcoming the Bottleneck of Data Scientists

Simon: "There’s a growing recognition that data scientists can often become bottlenecks in the insight generation process. Instead of focusing solely on R&D pilots, companies are now looking at how to scale machine learning across their data and business units."

‍

Sam: "Exactly. Scaling ML involves not just the deployment of models but also integrating them into business processes and decision-making frameworks. This approach helps in maximizing the impact of AI and driving more value from data across the organization."

‍

Integration with Databricks Across Industries

Sam: "Integration with Databricks is becoming widespread, even among traditional OT software and industries such as petrochemicals. Companies like Shell and BP are standardizing on Databricks, which is enabling them to realize the potential of high-potential use cases like predictive maintenance."

‍

Simon: "This trend underscores the versatility and robustness of Databricks as a platform. By providing a unified data and AI infrastructure, Databricks is helping these industries overcome legacy challenges and leverage modern AI capabilities more effectively."

‍

Addressing the Costs of AI/ML

Simon: "The growing costs of AI and ML are a significant concern for many organizations. JPMorgan and others have highlighted the need to monitor and optimize these costs. Databricks' recent release of their Predictive (Cost) Optimization solution, which promises to reduce storage costs by 50%, is a timely response to this issue."

‍

Sam: "Monitoring and optimizing costs is crucial as AI projects scale. By offering solutions that reduce costs while maintaining performance, Databricks is helping organizations manage their budgets more efficiently and ensure the sustainability of their AI initiatives."

‍

The Rise of Generative AI and Predictive AI

Simon: "With the rise of Generative AI, companies like Tangent Works are focusing on the evolution of Predictive AI. This shift is about harnessing the power of AI to not just generate content but also to make accurate predictions and drive proactive decision-making."

‍

Sam: "Predictive AI is indeed a game-changer. It’s about using AI to anticipate future trends and behaviors, which can significantly enhance strategic planning and operational efficiency across various sectors."

‍

Conclusion

By leveraging these insights from the Data AI Summit, it’s clear that the landscape of data and AI is rapidly evolving, with new tools and methodologies making it easier for companies to harness the full potential of their data.

‍