The Data Journey Diagram/Data River Diagram

Today, I am going to talk about a tool that I have just created. The tool is here for discussion, but NOT for re-use or re-distribution. Please use your conscience and do not re-use this tool without my permission, (which can be had for the asking).

The tool is called the Data Journey Diagram.

What

Graphical illustration to help users understand the usage of data generated by them, and to help BI designers see the most expected fallacies in their data.

For

Primarily, the tool is for the end users who generate data and the Analytics consultants who design the analytics solution for an organisation. The other stakeholders are the IT department, legal, security and line businesses.

How

The tool is very simple. We put the end users, the MIS team, and the data analytics team in one room.

Then we create a meandering river shape on the board. The Analytics Consultant puts a simple report at the end of the line. Then, he puts in all the data elements that go into making that report.

Slowly, each stakeholder who owns each data point, writes down where each data element comes from – all the processing points, and all the hands that the data element passes through. The river is used as a rough measure of time flow.

For each data element that comes from a different source, at the point of convergence, we draw a separate line to indicate a contributing tributary. Then the journey carries on backwards until we have a river system in place – indicating all the places where the data comes from, how it all comes together et al.

 

Ground Rules

At each stage, ONLY the stakeholder who owns the data at that point, will be allowed to create a “milestone rock” and name it. No one else.

 

Limitations

This tool does NOT:

A. Indicate dependencies.

B. Indicate And/or relationships

C. Indicate mathematical or logical operations. We know that 2 data elements come together, but we do not know if they use averages or totals.

 

What is achieved

At the end of this exercise, end users are able to appreciate, in very simple terms, how their data errors impact decision making at the CEO level.

The Analysts are able to see sources of the data being used in the high end analytics reports, and can talk to the stakeholders right there to find out if there are any anticipated data errors – time lag, fallacy, human error etc. These are VERY important inputs for the analytics design team, because GIGO is as true today as it was when computers were invented. 

So, what are your thoughts?

 

What the Everest teaches us about creating great organisations

If you want to trek up a hill, you can do it alone. And with some practice and grit, do it really well.

 But no one, and no one, can climb the Everest alone. To conquer a mountain, you must climb with a team.

The rules of the mountains teach much. There is no hierarchy. The respect is earned not by who pays whom, but by who knows what and who does what.

The Sherpa who, strictly speaking, is a paid porter, gets much respect. Because he has been up and down that mountain many times, and because he has spent his life observing this mountain, worshipping her. He understands her moods way better than you ever will. When he tells you to turn, you always do. Because he knows better.

What if we spent some time doing that in organisations? Listening to people who have been around a long time.. understanding from them why things failed when they failed, what happened in the past. . it will waste time and bind our thought process. But it will also teach us lessons that we dont need to relearn.

 I can remember a case study that we did at one of the courses. The details are not important, but the message was the same.

Do you think this would work for organisations? Why or why not?

* This post first appeared on my blog on ITtoolbox/toolbox.com on January 28, 2014

On Requirement Gathering

If you are a system design consultant, this piece is for you.

There is a children's poem that explains how the battle was lost for want of a horse shoe nail.

Take a printout of the poem and tape it just above your screen.
Remember that poem when you are doing requirement gathering. It is ALWAYS the small things.
****
When do you do the transaction - how long after you have done it in the real world? Why do you think that is the optimal time?

What do you look for when deciding your sales pitch? What are the elements you need at once place, together?

When do you think its ok to share your password with a colleague? For which apps will you never do it?

What will you do if your orgn KRA requires you to fill daily updates, but you are in a field area with no signal?
***
After 15 years, I have found that the level, industry, or technology does not matter. When a project is delayed or faces user resistance, very often, it is because we did not ask the human questions.

We did not understand the fine print of their lives.

At its core, requirement gathering is a human process. All our phenomenal technology - retina tracking, gesture analysis, only goes so far. In the Indian system, the importance paid to the individual, more than the 'system' is at the core of this human centric approach.  

As requirement gatherers, we should try to understand that this is a human interacting with a system, where the human is in charge. With that conscious shift in understanding, we will find ourselves approaching the discipline very differently. 

Shifting the consciousness to gratitude every morning

If you have noticed, before opening the store, a shopkeeper bends down and touches the floor of the shop. A performer touches the last step of the stage before getting on the stage. An auto driver starts the day by doing a small agarbatti or pooja in the auto.
Ever wondered why that is?
The shopkeeper, performer, and auto driver get their livelihood (rozi-roti) from the shop, stage, and auto respectively. Every morning, before they start work, they are thanking the means of their livelihood for providing for their families, and praying to it to bless them with a good performance.

There are some things that don't take a lot of time and effort, but cause a massive shift in our consciousness. The practice of thanking our livelihood before starting our work day is, I think, one such.

Wealth Generation in everyday life

One of the questions that I often asked myself is: How did Indian households accumulate wealth through literally centuries of Muslim rule. During this rule, Hindus were taxed at insane rates, and at many more instances than the non Hindus. The historical texts I read recently had stories of how the rulers prided themselves on systematically stripping the Hindus of all wealth, forcing them into penury and then incentivising conversion to Islam by giving monetary benefits.

Important Disclaimers
1. Please do not judge these rulers by the standards of today. Secularism, which is a given today in most modern civilisations, was a luxury in almost every part of the world until just 200 years ago. Remember the Protestant British crown and the impact on the Catholic church? These rulers were not being barbaric. The instruction of the time started by dehumanising all non-followers of a religion. In their minds, the missionaries of Christianity and the Muslim rulers were both converting these people back to humans. We have no right,  therefore, to judge them by our standards. Please refrain. All hate content will summarily deleted. My focus is on understanding financial and trade practices of India. We will stick to that mandate, using these conditions as the context within which these practices were used and how they managed to do some very important wealth generation and preservation.

2. This piece is a result of my research. Most of my sources are family practices, folk lore, and things head from practitioners. Indians have not, to my knowledge, documented their wealth generation and preservation practices. But, thankfully, they have kept their family traditions alive over hundreds of years. The belief system is still very well preserved. I observe and learn from that.

3. Every system has its pluses and minuses - if we judge. If we learn, they are just actions and consequences. My submission is - learn. Do not judge. How you apply these lessons, or whether they  can be made relevant today, is something we can decide for ourselves.


After much searching, am sharing some findings here:

• The family as one unit. 

The unity of the family is so inherent in the culture that even in Independent India, the Karta system is recognised as a type of economic entity - the wealth of the entire family is grouped together, and the expenses are done from the same core. As everyone lives together, expenses are pooled. Clothes are passed from one child to another. Ditto for toys. Skills are distributed, minimising people's dependence on external factors.

• Order of investment

In most Indian families, the first asset that is acquired, if not inherited, is land and real estate. A house, farming land, a shop - the type of real estate depends on the lifestyle and varna of the family. But the uniformity of the preference for this asset type is remarkable. After this, one invests in gold and keeps enough for impending weddings of the next generation. Then, other assets are acquired, if any. 

Do you know why that is important? The rate of inflation on these 2 assets is the highest. If they are secured, they give the highest returns, and one is able to invest at lower values. 

• Consistent small value purchases

At all festivals, a little gold or silver is bought according to the means of the family. Even if a family is very poor, at least once a year, they have to invest in metal as a form of religious ceremony. These small value but consistent purchases slowly build up the corpus of the family. In many trading families, there is a custom of adding one gold coin every Diwali. Imagine the value of that gold within 3 generations - 50 years. 

• The Hundi system and trade practices of pooling 

The Hundi system is well known. This system entails everyone contributing a small sum of money every month. The entire pool is handed over to one member of the group every month by turns. This way, everyone invests a small amount every month but gets a lumpsum perhaps once a year. This was not the only practice of pooling. Several trading communities, in particular, have a parallel system of pooling in parts and harvesting in bulk. 

• Temples as the custodians of wealth 

Almost all religions have the practice of making a small offering to God when we go to a religious place. This makes our religious places incredibly rich. In some South Indian states, the king, through assumed divine birth, is also the owner of the temple treasury. 

This serves many objectives: 

One, in the absence of state sponsorship , the temples could run religious events and ensure cultural continuity. 

Two, temples cannot be raided and looted as easily by rulers as individual homes. A single seth becoming too rich can be forced to part with his wealth by the ruler. But not a temple. 

 

If there are any other lessons, please do share. This is a very important area for us to learn from. 

We don't fix "the system" , we fix the person

If you have grown up with modern management theory, you are probably obsessed with "systems" - those magical things that ensure everyone has the same experience with an organisation.

Travel Management systems ensure that everyone travels according to their eligibility (read: Their place in the pecking order). Requisition systems ensure that everyone at the same level has similar access to corporate resources, and so on.

What happens when a person plays the system? Immediately, additional safeguards are put into place to ensure that the said breach does not happen again.

Iterations of this ensure that the system becomes progressively more rigid, harder to use, and so on. Until, of course, the system becomes a data entry mechanism and things continue pretty much the way they do on the ground.



Many Western employers find this exception seeking behaviour of Indian employees quite frustrating. It is as if we were born with a more than healthy disdain for following rules.



How do Indians manage uniform access and systems approach? 

We don't. For us, each individual is unique. One's access to resources is determined by their personal equity. Not by their place in the pecking order.

The Indian business system inherently recognises the power-authority equation, and combines both when determining the access level of an individual. Unlike the systems based approach, where only authority is recognised and power, though acknowledged, is not official recognised.

What happens when a person plays the system? 

In the Indian system, we don't fix the system. We fix the person. The person is called in to understand what happened, and why. After that, a suitable retribution is awarded, either publicly or privately, but such that everyone knows what happens to those who try to play the system.
And thus, everyone knows what to not do in an organisation.

Because the talent pool was limited and closely knit, what happened to one employee made it to the grapevine quickly enough. It acted as a deterrent. There was no need to "idiot proof" the system. We "tampering-proofed" individuals.

And that, I think, is a diametrically opposite view to correction management than the "Make our systems tighter" approach.

So, dear Indians and Westerners who wonder why Indians don't follow rules, please understand that our traditional systems give us privilege based on our power + authority. We inherently don't understand this One-Size-Fits-All method. This is also true of most high context cultures, where collectivism is so high that it is possible to easily bring errant individuals to compliance. Where each individual is under the spotlight so that the overall pattern appears perfect.

Your thoughts?

We don't hire a person, we hire a family

If you have worked in an Indian business family (including Reliance),  you would have noticed one or both of these things:
A. When an employee has an issue with the family (finances, child not listening etc) , they come to their manager (bade babu or seth ji), and a loan or a counselling session is arranged pronto. Sometimes, rules are bent to make this happen.
B. When a child graduates, the seth ji or bade babu calls the father and tells them to send the resume. The child is usually absorbed by the company.

You may also have observed this in domestic staff in India. We hire one person, then her relative joins as driver, another relative joins as cleaner, and the family is engaged. In villages, we are aware of entire generations of 2 families engaged in an employment relationship.

We're all in this together

If you come from Western business practices, you have possibly heard of a word - Nepotism. It is viewed as something negative. Something that takes opportunity away from a qualified person and gives it to someone who is friends or family with the decision maker.

So it will surprise you to know that in the Indian style of business, reference is the best qualification a person can have.

This is practised in two ways

A. We don't hire a person. We hire the family. 

During the industrialisation era, industries were built in the middle of nowhere. It was, therefore, important to build townships near the factories also. This led to bonhomie among the families of those working together.
Contrast that with Electronic City in Bangalore, where employees are expected to spend upto 4 hours a day in buses.

Whenever there was a company event, the entire family was invited to it. The manager made it a point to speak to the wives and convey his compliments. When a person got an award, usually, his wife and family were asked to come up to the stage with them to collect that award.
Contrast that with today, when the families go to ONE family day a year and all award functions are usually held without the family in attendance.

B. Other things being equal, we hire someone we know. 

This was EXTREMELY important. This is why the entire social and religious structure of India is based on professions. Each profession was specialised. Training began early, and by the time a person was ready to join the workforce, they had already had over 10,000 hours of real training on it.

Even when education went from vocational to professional (read: 3R and formal schooling based), companies continued to hire from the family pool of employees if possible. When a child graduated, the father came first to Bade Babu to see if a vacancy was available in the same company.

Surat diamond industry labour is always hired from a certain community, and all the families are interlinked. So one individual cannot even dream of stealing and getting away with it.

The family sort of hangs out together

Why was this done? 

I am, of course, only hazarding a guess. 

1. It ensured employment stickiness 

Just like in story telling, we engage as many senses as possible, to make the experience richer for the audience, in employment, we need to ensure stickiness. After work, the person goes back to their family. If the family is also engaged with their employment, the employee's stickiness goes up many notches. The decision to leave is not an individual one. The wife has friends, the kids have friends, the boss's wife knows your wife. Its a complex web that one needs to extricate oneself from. Much harder than a resignation to one boss or even looking out for a job. 

2. Oh, the beauty of reference checks! 

Today, professional organisations struggle to do reference checks. They get identity proofs , they check educational qualifications online. And they still dont get relevant, qualitative information like whether the person has been corrupt or a predator in the past. Has the person demonstrated any domestic violence tendencies? Was he a bully in the previous organisation? 

In this method, which sustained for thousands of years before it became possible to do formal ref checks. A person would only recommend another if they were absolutely certain of their ability to deliver and their integrity. One puts one's personal credibility at stake. Its personal. 

Companies today try to hire through employee referral programs. Some of them are successful. However, in some key points, these programs differ from the Indian method of hiring who we know. 

For starters, the engagement level of the referrer is not so high that they feel that they are staking their personal credibility. Its a purely professional transaction. 

Secondly, the referrer and the referee are usually not allowed to work together, which beats the entire purpose of referring. Why would I want to bring my friends into the company if I can't work with them? 

Conclusion 

Was it the best way to hire? 

We don't know. But unlikely. There is no one best way to do things. We do what works for us. We learn from multiple sources. This is one of them. 

Is it applicable today? 

Depends. On the industry, company, and the organisation culture. 

Should i use it?
Not without adequate checks and balances, and training. Indian business practices cannot be applied piecemeal. They have to be understood deeply, imbibed, practised in everyday life, and then used as a specific method in business. Business empathy comes from personal empathy. We first spend years being empathic in our homes, putting people first. Then, when we listen to an employee, we do that with genuine concern, not to put a tick on the HR checklist.
I leave you with that thought.