Category: Uncategorized
-
Breaking Down Silos: Overcoming Turf Wars and Politics to Streamline Data Infrastructure
This is one of the biggest problems I encountered when helping companies build their data-driven analytics transformation. I have been helping companies solve this particular problem for more than ten years already, and I thought that by now, we should have gotten over this problem already. Unfortunately, the turf war problem is even worse in…
-
Enhancing Collaboration: How Data Teams and IT Teams Can Work Together Effectively
I have many students whose backgrounds are in IT, so I have heard many horror stories about the problems they have with the data team. I also heard similar stories from my students in data teams who have to work with the IT team. In today’s data-driven world, organizations increasingly rely on seamless collaboration between…
-
Essential Components and Infrastructure for Effective AI System Implementation
“Jack, a director at TechPR, was thrilled by the potential of artificial intelligence after reading several articles praising its transformative capabilities. Convinced it was the perfect solution to streamline their customer service operations, he hastily decided to implement an AI system, expecting it to be a straightforward process. Confidently, he instructed his team to proceed…
-
The Disconnect: Strategic Planning Without Data Expertise
As a technical advisor, I have witnessed this scenario happen a lot in Asia. In today’s data-driven business landscape, integrating technical insights into strategic planning is essential for maintaining a competitive edge. However, a significant problem arises when upper management lacks data technical skills and makes strategic decisions without incorporating the input of data engineers…
-
Letting Data Engineers and Data Scientists Roaming Free in Strategic Planning.
Lately, I have attended several data meetups in Asia, and I have observed a problematic situation. Many upper management are letting engineers and technical people run the company’s technical strategy. This scenario is understandable as many upper managers need to gain new data technical skills and must rely on technical people for the issue. The…
-
Data Mesh in Higher Education: Navigating the Challenges and Complexities
My last story was about empowering Universities with the data mesh, such that faculty can easily build data products that students and upper administrators can consume. Unfortunately, this is rather complex and very challenging, and your data mesh project can easily turn into a mess. While the concept of Data Mesh has gained traction in…
-
Empowering Academic Excellence: The Case for Data Mesh in Higher Education
I had a vision that every faculty would have data products where these products are geared toward providing a better student experience. However, getting the project through upper management is not an easy task, and I have the experience firsthand. Convincing a university president to invest in a Data Mesh project involves presenting a compelling…
-
The Challenge of Building Effective Data Teams in Modern Organizations
One of the most difficult tasks that I have experienced as a data consultant is helping companies build an effective and economically sensible data team. The main reason for this difficulty is that companies do not understand the importance of building a data team systematically so that the team can grow and scale with the…
-
AI-Driven Innovations for Superior Customer Experience
I have seen so many companies in Southeast Asia trying to get on board the Gen AI and ML train. Unfortunately, many companies are not choosing appropriate applications on which to focus their Gen AI and ML resources. In this article, I will focus only on using Gen AI and ML to improve customer experiences…
-
No Ice for you. When Not to Use Iceberg: A Critical Evaluation
I love Ice, especially Iceberg, which helps simplify data management and enhances query performance without requiring users to manually manage partition metadata. Iceberg also provides transactional consistency between multiple applications where files can be added, removed, or modified atomically, with full read isolation and multiple concurrent writes. Given the recent hype that Databricks and Snowflake…