Data Inconsistencies Are Everywhere
Much like humans, data have their own quirks and shortcomings; as such, I knew to expect occasional missing or invalid values in big datasets. After all, I have perpetuated this myself every time I filled out a registration form omitting my personal phone number or entering a phony email. Today, I analyze millions of records entered by hundreds of users and I see how quickly data issues can accumulate. Phone numbers, date formats, and state abbreviations can be entered differently by every user. These kinds of issues are only the tip of the iceberg. My first several weeks at Premier were spent learning the Applaud® software and applying this tool to complex data challenges.
How to Explore a Data Landscape
One of the most satisfying new phrases I learned at Premier is “data landscape.” The data landscape of an organization is the set of systems, databases, and applications it uses to carry out business. I love the phrase because it encourages me to think of IT (Information Technology) systems as something to be explored and understood. Premier is teaching me to approach data landscapes as a explorer: common data structures are my guideposts, industry standards are my North Star, and a successful go-live is the treasure I am looking for!
What is a Mapping Specification?
Part of being an ‘data landscape explorer’ is creating a map. Over the course of a project, I will learn the location of thousands of data points, the meaning behind a multitude of code values, and the source of millions of transactions. Part of my job at Premier will be to document where specific information comes from, where it will go, and how it needs to be transformed along the way. I will reach this deep understanding of data through collaboration with my colleagues at Premier, the client organization, and our partner system integrator. The end result of our collaboration is a mapping specification document which outlines how the existing data will need to be manipulated to be compatible in the new system. This specification is the foundation of conversion code I develop to power the necessary data transformations.
Metadata is Awesome
It is impossible to play a game without rules. Metadata provide the ground rules for a database and allow me to know data structures and types without looking at specific records. With Applaud, I automatically gather metadata from across my client’s legacy IT systems and am able to incorporate it into a central data repository and data dictionary. This data dictionary allows me to understand critical information such as key structures, indexes, field types, field lengths, and the relationship between different data tables.
Testing Does Not End After College
I have a degree in neuroscience. Naturally, I value the importance of testing as it applies to the scientific method. The most remarkable parallel between my university studies and my current career is the amount of rigorous testing applied to data migration. Internal communication is swift, and our team zeroes in on issues with surgical precision. The team is always working together to report errors, building processes to resolve issues, or updating conversion code to account for new discoveries. It is critical for me to be a good steward of client data, which means that everything must be tested. The data that passes virtually through my hands may drive procurement for a hospital, meet payroll for thousands of employees, or drive budgets for global conglomerates – it must be correct.
Never Forget Where You Are Going
It is critical to understand that data migration is oftentimes just one component of a larger initiative. As I learn more about my client and the target system, I pay close attention to what the organization’s leadership cares about. Is improving customer relationships a goal? Does the organization want to accelerate growth by acquisition? Does the financial department need to be overhauled? These are the questions I need to ask to truly understand where this organization is going and how digital transformation can help them get there.
Get Messy, Have Fun
My teammates at Premier have been welcoming and understanding. My fellow consultants realize that data is messy, relationships within and across databases are convoluted, and unpacking a data landscape requires a great amount of mental energy. It feels like running hurdles with my brain. But I am not doing this alone! I have the wind at my back and an entire company cheering me on. Even leadership takes the opportunity to get to know me and help me succeed. The consultants, managers, and leadership make themselves available to answer questions and share time with me. I look forward to the many lessons I will learn, the go-lives I will celebrate, and the personal and professional growth I will experience.