MDM system without primary normalization. Actual approach to solving old problems
The mess in the directories storing the basic data of the company is a common problem that causes quite a few troubles, among which there is confusion in the primary documents due to incorrect establishment of the nomenclature or the purchase of goods that are in fact in the company's warehouses. This can be avoided by implementing the MDM system and normalizing the data.
Normalization eliminates directories from unnecessary information, makes entries in them uniform and standard. This allows you to use high-quality and structured data, take them from a single source and be sure that they are correct.
Let's look at how it is customary to normalize directories storing basic company data, and how to avoid an expensive and time-consuming process.
For example, take the directory of material and technical resources (Nomenclature). The normalization of such a directory requires a lot of time, money and experts to clean up the data.
What is normalization?
Normalization is the reduction of information in directories to a standard form, excluding duplicates and obsolete entries, supplementing (if possible) incomplete positions.
It is assumed that after normalization, your directories will store up-to-date and consistent information that employees of the company can use to carry out their work together.
How to normalize now?
Normalization of directories is classically carried out in two stages:
- The contents of the source directories are processed before being loaded into the MDM system. This procedure is called primary normalization.
- After loading into the MDM system, entries in directories are added or changed according to the normalization technique adopted by the company.
What are the disadvantages of the described approach?
The main drawback of this approach lies in primary normalization.
Suppose a company maintains a nomenclature reference book of 200,000 entries. To conduct an initial normalization of such a volume of data, 20 experts stand out before loading into the MDM system.
Based on the conditional norm of processing records by one expert at 1,000 positions per month, we get that such a number of records can be normalized in about 10 months of work.
We will take into account in our calculation that the experts are not robots, but people, and add to the processing period another month and a half for possible force majeure, illness, holidays and unforeseen situations. We get that the source directory will be processed by 20 experts in about one year.
It is important to note that this year the guide will be updated with new entries. Their normalization will also take time. So, the period of primary processing of directory entries can stretch for another months.
At the same time, users will be able to see the correct entries only after the system is put into operation. At the same time, the disadvantages of the normalization will begin to be identified, which will require additional processing of the contents of the directory.
How much will you pay for the initial normalization?
The average price of the initial normalization of one entry in the Nomenclature directory in Moscow is 150 rubles. In our example, 200,000 entries. The initial normalization of the data of such a directory will cost 30 million rubles.
And here you need to remember that with the usual approach, you will pay the initial normalization of all directory entries: current and not used by the company.
Why continue to carry out primary normalization?
Initial normalization is carried out in order to obtain fully normalized records of the source directories. However, often after the initial normalization, about 40% of the generalized and inaccurately described data remains in the original directory, which are recognized as not normalizable and are loaded into the MDM system in their original form.
In the case of the “Nomenclature” reference book, such records can be clarified only when the nomenclature is verified by experts in the company's warehouses by appearance or technical passport (inventory). That will add work and take a lot of time.
How to normalize directories more profitable and efficient?
- Exclude primary normalization.
- Analyze directory entries to determine groups of entries by the degree of importance and relevance of the data.
- Identify possible peak moments for updating records. They can be centralized procurement campaigns, which are carried out at regular intervals.
- Download the source directory to the MDM system.
- Block from changing part or all of the directory data.
- Update blocked data gradually at the request of users.
With this approach, all the raw data will get into the MDM system. They will be normalized gradually after loading, which will allow:
- Put the system into operation in a short time, without waiting for the completion of the initial normalization of the directory.
- Normalize what employees actually use in their work and do not process obsolete records.
- Do not spend money on the work of experts on the primary normalization of the directory.
In this case, the percentage of the presence of incompletely described data will be reduced to zero. Such accuracy is achieved by the fact that when updating the data of the directory on the user's applications, experts will be able to clarify all the data from the author of the application.
How does it work on the example of the "Nomenclature" directory?
We remember that we have a directory “Nomenclature” of 200,000 entries. We do not carry out its initial normalization and load it into the MDM system in its original form. Already at this stage, we do not spend about a year on the initial processing of records and on the payment of the work of 20 experts. In this case, we have several options for normalizing it:
- We analyze records and divide the entire amount of data into groups by importance and price.
- After loading, we block for changing the most popular and expensive part of the nomenclature. Now, for the purchase of these positions, company employees must fill out applications.
- When processing applications by staff experts, a blocked group of directory entries will be gradually normalized, and then similarly, the rest of the nomenclature group. Locking of directory entries will not apply to write-off operations, which will allow working with data in the normal mode.
- Without the user's request, the blocked items of the item will not be updated. Such items will be transferred to the archive according to the rules of the company: during the next inventory or after writing off the balances on them. Based on experience, from the entire array of 200,000 records, only part of the truly relevant and used positions will be normalized (as a rule, about 20% (40,000) of existing records).
The procedure can be carried out differently - by blocking for change all the entries in the “Nomenclature” directory, and not individual groups. Thus, only normalized entries will gradually appear in the directory, and irrelevant items will also be identified and transferred to the archive. But in this case, there is a risk of uneven loading of experts in processing records during peak periods of procurement of items. This must be taken into account when conducting analytics and correctly determine the load of experts.
Based on practice, when using the new approach, the Nomenclature reference book will be normalized on average for one year. This time is usually spent on the initial normalization of all entries in the directory - both needed and outdated. Therefore, the normalization deadlines will remain the same, but it will be carried out without unnecessary costs of money, experts and delaying the date of commissioning of the MDM system.
Thus, with the new approach, you will get everything that you expect from the initial normalization: up-to-date and correct reference data, which are used in everyday work, but at a lower cost. At the same time, the saved money can be spent on the development of classifiers, name standardization, development of system functionality and other improvements, which I will discuss in the following articles.
Do not think in a stereotyped way: study current approaches to solving old problems that will save your money and time!