Posted on

Meditation #1 Hire A’s?

“While A’s tend to hire A’s, B’s tend to hire not just B’s but C’s and D’s too”

From the section “The herd effect” in the book How Google Works by former CEO of Google Eric Schmidt and Jonathan Rosenberg

It is unclear the precise meaning of A, B, C and D, but from the context it can be gathered that it is a categorization of employees where the quality is descending with every letter of the alphabet. Presumably it alludes to the American grading system. This echoes Steve Jobs’ talk about always hiring an A-team and indeed I would think this is more a generic Silicon Valley insight than a Google thing. It seems to indicate that there is a superior class of employees that you need to attract and that the rest is bad that will make your company even worse. 

Before we start to evaluate the merits of the statement, we have to check the assumption that employees can be put into squarely delineated quality brackets. First question is how you measure quality of employees. The discrete labeling seems to indicate two important assumptions: 

  1. that this pertains to a person in general, not some particular area of expertise of that person. You are either an A or you are not 
  2. Another assumption is that the predicate is immutable. If you are an A you always were and always will be an A

These assumptions indicate that we are working with the philosophical position of essentialism, the view that an entity has an essence, from which its behavior, appearance or traits can be derived. In psychology this is used to describe how humans have a tendency to conceptualize biological entities and humans according to an immutable essence. Based on this essence it is possible to deduce behavior for other members of the same biological class. 

While essentialism may be a common human trait it does not mean it is the best way to conceptualize other humans. The root of racism is also derived from essentialism, and we don’t just blindly accept that as a viable or helpful way of assessing the merits of other people, so why should we accept this piece of Silicon Valley wisdom at face value? 

We should not. Because it is wrong. Let us look at the two assumptions again: 

The first assumption stipulates a general level of quality for a person but there is no reason to assume that a person can be A level at all traits. If not for anything else, then for the very fact that some traits are mutually exclusive. If we think about it in terms of physical qualities, it makes no sense to talk about A athletes across the board. An A weightlifter will be an F marathon runner and Vice versa. An A level football player may, however, be an A level baseball and basketball player and this is what we often think about, when we call someone a great athlete.  There are examples of such great athletes that have competed at the highest level in the NFL, MLB and NBA. But looks are deceiving here. These sports are only superficially very different. They are built around explosive outlets of energy, eye hand coordination with a ball and little stamina. It is less common, if it ever happened, that an elite athlete moved to the NHL even though it is similarly explosive, because you suddenly need another skill, that is, skating. This great athleticism will not either apply to swimming or to bicycling. 

You can also counter that in track and field there is nothing but general athletic ability. Look at Carl Lewis who won Olympic gold medals in many different disciplines. Again, looks can be deceiving. He competed and dominated the following disciplines: 100 m, 200 m, 4 x 100 m relay, long jump. These are ultra-explosive and none of them takes him out running further than 200 meters. How would he fare in 400 m, 800 m, pole vaulting, discus or 2000 m? We don’t know since he never competed. My guess is that he wouldn’t be an A athlete in these and probably an F in pole vaulting. 

In the tech industry there are similar complications. You cannot be both adventurous and want to try new things and risk adverse making sure that everything works. If you are working on quantum computing, you probably have a pretty high tolerance for failure and appetite for risk. If you are developing new models of airplanes you probably (and hopefully) don’t. The A person in the quantum computing setting may very well turn out to be an F- person in the aviation industry. 

Can-do attitude and perfectionism also do not align. The employee who is ready to approach any job with a pragmatic mindset and get things done will succeed in a climate of constant change, such as a startup, where you don’t know what you will do tomorrow or even later today. That person would probably not fare well in a heavily regulated industry like banking. The perfectionist though may thrive in a setting where work needs to be done with acute attention to detail. Switch these two persons around and they will no longer be A’s

The second assumption, that you will remain the same, is similarly ill founded. First of all, human cognitive abilities develop and change over time. In mathematics and physics there is a tendency for people to peak in their twenties. Einstein, Tesla, Newton and Leibniz did their most impressive work before they were 30. Conversely, with age comes greater ability for synthetic thinking: few philosophers or historians peak before they are 40. Similarly, politicians have a tendency be more successful when they are older. It takes time to build up the skill to interact with people to achieve a result. It also takes time to build alliances and network. This is not an immutable trait.

Another more mundane concern in Silicon Valley is burn out. Even the best, or maybe in particular the best, programmers sometimes burn out, and are not able to write any good code anymore. Others just do not stay on top of development. They may have been the smartest assembly coders in the room but just never jumped on this newfangled thing called C++. They would hardly be considered A’s today. On the other hand, some people continue learning and may not have started out on the right path but changed to become better. Steve Jobs himself started out in liberal arts and learned tech skills only later. He would probably never have been hired out of college by Google. 

Consequently, what we can deduce is that quality is always domain specific. There are no A people per se. They are always high quality with regard to a particular area of specialization. 

We can also see that quality is not immutable. Even the best people turn bad for one reason or another and even bad people can become good. People change both according to a biological and cognitive development and due to personal circumstances. 

It is consequently dangerous to assume that A’s will magically beget A’s in a continuous stream of awesomeness. A’s burnout and A’s sometime don’t adapt. They degrade. Following the advice could therefore lead to a false sense of confidence. Classifying people as A’s can also be dangerous if you put them too far out of their area of expertise. Many companies have seen how the brilliant engineer turns out to be a subpar manager. Engineering’s attention to detail and focus on there always being a right and a wrong is perhaps not always conducive to employee empathy and development. This line of thinking also creates missed opportunities. If a person has historically been given the C stamp and that is all we look at then how will we ever know that this person developed into an A? 

A further point concerns that of generalizability. It is fine for Google to hire only A’s but most companies are not in a privileged situation that Google is and cannot attract any of the best. We have to remember that Google and the top Silicon Valley companies are in a unique position where they earn so much money that they can offer whatever compensation. They have also made a name for themselves with prospective employees. That means that their problem is one of filtering. Everybody wants to work for Google, their problem is to find the best. 99,99% of other companies in the world do not have that problem when it comes to recruiting. Rather ordinary companies’ problem is one of attraction. For example, one of the thousands of auto-parts suppliers will not be known to most potential applicants. Therefore, they have to attract not filter employees. If they can even get somebody qualified, they would be happy. Talking to them about hiring only A’s is close to an insult. They would never be able to because they don’t have infinite pockets, Michelin chefs in their cafeterias and 20% time for the employee to work on what he or she thinks is fun. The vast majority of the world’s companies fall into this category of unknown companies, with limited budgets and a regular workplace with a kitchenette and a water cooler. 

The last point is more subjective. The sentence seems to echo privilege and entitlement. Who are these A’s? They are the best people from the elite universities in the US: Stanford, MIT, Columbia. They were able to become perceived as A’s because they got into those universities. Some do get there due to hard work and scholarships. Most don’t. They get there through their parents’ wealth. Google doesn’t go to a Southern community college or African universities to look for A people. They go looking where the managers went themselves. 

As can be seen from the above, not only is the sentence wrong and unhelpful, it may be dangerous to follow even for Google. For the vast majority of companies, it will be completely irrelevant if not downright insulting and it tacitly expounds an air of privilege and entitlement that they overtly claim to be fighting. 

Consequently, I would like to turn the sentence on its head. Since most employees are not A’s according to the measurement scale of Silicon Valley, we need to think of how we make the most of the B’s and C’s and D’s. This is the real problem for the world (not for Google and Silicon Valley). How do we get the best performance out of the people who prioritize being with their kids or family, the people who prefer hanging out with friends or playing tennis to working 80 hours on the latest feature that may be gone next month? These people would never be perceived as A’s that will invent the next big thing. But most companies don’t need that. They need happy reliable people that do a job within a limited scope well enough. How do we find the person with the right skills for a particular job? They need people with new skills but can’t hire them, so how do we train and create the environment for ordinary people to perform new functions? And last of all how do we turn the story to redeem the dignity of the people in the tech industry who go to work to do a solid job 9 to 5 without any fanfare? 

These are the real problems that we need to be focusing on in order to take advantage of technology in the future and create a better world with more productive and happier employees. 

Posted on

Move Fast and Do No Harm

The advent of SARS-COV-2 has mobilized many tech people with ample resources and a wealth of ideas. A health crisis like this virus calls for all the help we can get. However, the culture of the tech sector exemplified with the phrase “Move fast and break things” is orthogonal to that of medicine exemplified with the Hippocratic oath of “first do no harm”. What happens when these two approaches meet each other? Unfortunately, the well-intentioned research and communication sometimes results in the trivialization of scientific methods resulting in borderline misinformation that may cause more harm than good. 

Moving fast

Under much fanfare the following piece of research was presented by Sermo, a social network platform for medical professionals, on April 2nd: “Largest Statistically Significant Study by 6,200 Multi-Country Physicians on COVID-19 Uncovers Treatment Patterns and Puts Pandemic in Context” . This is a survey of what doctors are prescribing against COVID-19. So far so good. This would indeed be interesting to know. But already the next line sends chills down the spine of any medically informed person: “Sermo Reports on Hydroxychloroquine Efficacy”…Can you spot the the dubious word here? Efficacy? Let’s rewind and rekindle ourselves with what efficacy means: “the ability, especially of a medicine or a method of achieving something, to produce the intended result” according to the Cambridge dictionary.

It gets worse:  The CEO claims:

“With censorship of the media and the medical community in some countries, along with biased and poorly designed studies, solutions to the pandemic are being delayed.”  

What he means to say then is that the more than 400 clinical trials that are already under way are one and all “biased and poorly designed”? Criticism is always welcome because it sharpens our arguments and logic unfortunately the piece does not have one reference to even one study that would exemplify this bias and poor design. 

This is the first clue that this is a tech person and not a medical or scientific, I would say not even academic person. This is a person that moves fast, throws out unchecked assumptions and accusations and then moves fast to his much better designed study that was presumably scribbled equally fast on the back of a napkin.  

This is where clue number two becomes evident.  What is this superior method that is above the entire medical world scrambling to produce scientific knowledge about the outbreak and efficient therapies? Naturally the inquisitive reader is drawn to the sentence: “For the full methodology click here”. I click and read with bated breath. 

We are informed that the survey is based on responses from doctors from 30 countries with sample sizes of 250 respondents. Sounds fair although 30 times 250 is 7.500 not 6200 as mentioned in the title (what happened to the remaining 1.300)? We are told that the Sermo platform is exclusive to verified and licensed physicians. Let’s pause here. How is it exclusive? This is the methodology section, and this is where you tell me HOW you verified the doctors. Otherwise I have no idea whether the results actually mean anything. It could be a mixture of neo-nazis and cosplay enthusiasts for all I know.

Next we read :

“The study was conducted with a random unbiased sample of doctors from 30 countries”.

That’s it. For people unfamiliar with the basics of clinical scientific method this is the equivalent of a suspect in a trial getting up in front of the judge claiming “I totally didn’t do it. Just let me go”. Again, how do we know? Maybe you sent your list of invitations based on a secret list from Donald Trump with doctors that are fanboys of Cloroquine? Maybe the responding doctors are unemployed (for a reason), which would explain why they had time answering the questionnaire. What was the distribution of age and gender? Was it representative of the age and gender distribution of the country they come from? Traditional scientific studies based on samples like these can dedicate up to a third of the article just to demonstrate that indeed there was no bias. Here we are offered one line without any evidence.

The study was based on a survey that took 22 minutes. Basically, any Joe-Never-Was-A-Doctor could have done this with Survey Monkey and a list of emails scraped from the internet. That is also fine, but we don’t get any information about what the questions were. Next section is the “Data Analysis” (and remember we are still in the methodology section) informs us that all results are statistically significant at a 95% confidence interval. Why was 95% chosen and not 99%? What were the actual p values?

In a little less than a page we learn virtually nothing that would help us ascertain the validity of the reported results. And where is the discussion? Could it be that the preferred treatments were dependent more on local availability than choice on the part of the doctor? Was there a bias in terms of geography, gender or age in relation to what they prescribed? Did everyone respond? Was there a pattern in those who didn’t respond? 

Although we are left with a lot of unanswered questions, the attentive reader can already from this very sparse information deduce a damning flaw in the study design that completely undermines any of the purported claims to efficacy: the study asks the doctors themselves about their treatments! Now why is that a problem?  Doctors, like all humans are susceptible to confirmation bias. This means that they are prone to look for confirming evidence. If they prescribe something, they have reasoned is a good drug they will look more confirmation that this is indeed a good drug. This is exactly why any proper designed study that shows efficacy needs the administering doctors to not know what they are treating their patients with. This is why the double blind test is a hall mark of any demonstration of efficacy. 

Where do we go from here?

I am from the tech sector myself and not trained in medical science (although I have taken PhD level courses in advanced statistics and study design), so don’t get me wrong, I believe strongly in the power of technology and I want tech people to engage and help as much as possible. However, as should be apparent by now, this is not helpful.

Had this been presented as what it is, a descriptive survey of what doctors are prescribing against COVID-19, it would have been fine and even valuable. Rather it is pitched as a revelatory study that undermines all current research, something it is not, which may undermine the serious efforts being undertaken currently to find adequate treatments. The clear favourite of the article is Chloroquine, but Chloroquine poisoning has already cost lives around the world due to the current hype. Recently an Arizona man died after ingesting Chloroquine on the recommendation of President Trump. How many more will die after reading this flawed and unsubstantiated “study”?

This is where the “move fast and break things” attitude has to be tempered with the “first do no harm” attitude. When tech people who know nothing about science or medicine use their tech skills they need to openly declare that they do not know anything and that this is not peer reviewed and only subjective opinion. Present the interesting findings as what it is and do not ever make any claims to efficacy or superiority to the medical system of producing knowledge that has increased our global average life expectancy from 30 years to more than 70 years in the past century. 

Tech people should still engage but they should stay within their sphere of competence and not trivialize science. Scientists and medical professionals don’t correct them on software design or solution architectures either. So, please don’t get in their way. 

Let me then give an example of how tech people should engage. The folding at home project simulates how proteins fold and thereby help the medical community in possible drug discovery. It has succeeded in constructing the world’s most powerful supercomputer delivering more than an exaflop, that is, more than one quintillion calculations per second . It works by letting people install some software on their computer and thereby contribute their compute power in a distributed network of more than a million computers worldwide. This is a good model for how tech people can support the medical community rather than undermine it. 

We in the tech sector need to move over and support our friends in the medical world in this time of crisis and approach their world with the same respect and caution that we expect others to show our domain of competence. Even though we are extremely smart, we are just not going to turn into doctors in a few days. Rather than move fast and break things we should “move fast and do no harm”.

Posted on

How truck traffic data may detect the bottom of the the current market

It seems evident we are on our way to a recession . This will prove a challenge for many, and our world economies will suffer. Not least the stock market. We are similarly probably headed for a bear market.

But the stock market is ahead of the curve and typically turns around about 5 months before the recession ends. For investors it is therefore important to look for indicators of when the current bear market is turning around since no one wants to invest in a bear market. 

Since this is a unique situation we haven’t been in before we need to look for unique indicators. It has been suggested  by Supertrends Institute that we should look for numbers such as number of new cases, new hospitalizations and number of deaths to start declining but since the world treats this very differently across nations it may be difficult to find out what countries to look for or whether to look for the total. Looking for the global decline may be misleading since the economic impact of countries may differ. A massive outbreak in Venezuela may cloud that view, since the economic integration of that country is not significant. 

Furthermore, there may be a lag between this point and when people actually feel comfortable going out. Consequently, we should look at other more robust indicators. One suggestion is looking at what can be inferred from traffic data. But should we just look at google/Waze data or telecom data to tell us the raw volume? That would be an indication of when traffic starts to pick up again. That is true. It is also however a data source with severe limitations. First of all, none of them have a complete view of traffic. Google and Waze only monitor its user base and can apparently easily be deceived as was recently demonstrated in Berlin. Telecoms only know what goes through their own networks not their competitors’. Second, all of these data sources know nothing about what sort of vehicle is moving. From an economic point of view, it makes a big difference whether the movement comes from a person in a bus, a car, a motor cycle or a truck, since trucks are reliable indicators that goods are moving around. All the others are not. 

It is not enough to look for a surge in traffic in order to spot a turnaround in the economy., This could be motor bikes or cars. What we are looking for is a surge in trucks, since trucks bring goods to stores and only when stores again receive goods will we know that people started spending. 

None of the existing solutions actually tell you what goes on in traffic. This is why we developed sensorsix to monitor not only traffic flow but also the composition of traffic. We monitor at the number of different types of vehicles at any given time through a network of traffic cameras. 

Cars and trucks on Zealand March 2020

The effects can be seen pretty clearly. One example is how the traffic quickly fell after Denmark was put on lock down. This figure shows the volume of truck and car traffic on Zealand in March. On the evening of the 11th Prime minister Mette Frederiksen announced that all public workplaces would shut down and employee s work from home. On the 13th borders closed. This resulted in a significant drop that echoes the decrease in demand due to the lock down of restaurants, cafes and most stores. While it was not illegal to drive around it is clear that truck traffic dropped much more than car traffic. IF we were just measuring the total volume of traffic that may not have been apparent. 

Another example is from New York where we measured traffic in the whole city. Here is an illustrative sample from December. 

Trucks in New York City December 2018

We can see a lot of truck traffic in the days leading up to Christmas day right until the last day where people are shopping. Then again after Christmas we see a similarly high number of trucks presumably carrying returned gifts, but then traffic is levelling off the rest of the month all the way down to the level of Christmas day because of the sudden decrease in demand. 

 These are just illustrative examples of the correlation between truck traffic and demand. We would expect to see a surge in truck traffic when the economy of our cities are really picking up and not until then. 

Posted on

Using traffic data to understand the impact of COVID -19 measures

We at Sensorsix have built a tool for ambient intelligence. Ambient intelligence is knowledge about what goes on around us. In our case it is built on what we can learn about human mobility from sensors. We have been in stealth until now working on a prototype to quantify the flow of human movement in particular traffic. Basically we use Machine Learning to extract information from video feeds to measure the volume of vehicles, pedestrian bikes etc. across time on select locations.  

As part of our testing of the product we had set up monitoring of the region of Zealand in Denmark. For those unfamiliar with the geography of Demark, Zealand is an island on which the capitol region of Copenhagen is located. The region is home to almost 2,3 million people. We wanted to understand the ebb and flow of traffic, the heartbeat of the region if you will. 

We started this test on Sunday March 8th. On the evening of March 11th the prime minister of Denmark, Mette Frederiksen, closed all schools and required all public employees to work from home. Most schools and institutions closed down already the following day. On Friday the 13th at noon the borders to our neighboring countries were closed as well. Since Zealand is next to and deeply integrated with Sweden these two events would be expected to have a significant impact on mobility in the Zealand region. 

Since we were monitoring the traffic from before the decision, we are able to accurately quantify and visualize the flow of traffic. The figure below that displays traffic volume from noon Sunday March 8th until noon Sunday March 15th. For reasons of simplicity we chose to focus on cars, so the figure only displays cars. Different patterns may exist for other types of vehicles, but the majority of traffic is cars. 

When we look at the pattern, what we see is the usual heartbeat of a city. Previous research and our own pilots in New York have shown the same pattern where traffic increases in the morning, has a noon dip and then rises in the afternoon and evening. But it is clear that even if the pattern is recognizable, the heartbeat is losing power. Just how much may be clearer from the figure below. Here we see a jaw breaking drop of about 75% in traffic volume. 

These are just some preliminary findings that we wanted to share for reflection and in the service of public information. Based on our data we can see that this is not a drill! It is not fake news. It is not tendentious journalism finding a deserted or heavily trafficked road depending on what they want to see. It is not exaggerated and it is not played down. it is a 75% drop regardless of how you frame it. In these times of fake news it is all the more important to get solid facts on the table. This is exactly what we built sensor six for. In all modesty we are probably the only ones in the world who can tell what actually goes on in traffic. 

What can this be used for?

A fair question is therefore what we can use this data for. Is it just another piece of data to throw on the heap? We think not. In the current Corona context, there are at least three key issues that solid ambient intelligence can help solve

Compliance – do people really stay at home or do they ignore the orders political leaders are giving them? This is an interesting way to provide a fact-based way of monitoring the efficiency and compliance of curfew and other measures of limiting trafic. 

Efficiency –  since this is a good proxy for degree of  quarantine a society is enforcing it is potentially an important metric. The frequency of interaction between people is an important variable in the spread of an epidemic disease and understanding the trends in mobility will give an indication about what that is. We should be able to correlate with the effect on number of infections and morbidity in the longer term. Obviously the effect will be delayed.

Economic activity – it should be possible to correlate the flow of traffic with economic activity. Initially it will of course be a drop and similarly the effect will be delayed. We can use the data to understand the economic impact a drop has. Eventually it should turn around and the rise in traffic volume should be the first harbinger of increase in economic activity. 

We will keep monitoring the traffic and supply other interesting insights that we can mine from our data. 

Note on methodology: we are continually monitoring roads leading to all entry points to and exit points from Zealand, which means all bridges and major ports. All traffic that comes into or goes out of the Zealand region is quantified. Based on this data we generate a volume score that is tracked continually

Posted on

Why Your Organization Most Likely Shouldn’t Adopt AI Anytime Soon

Recently I attended the TechBBQ conference. Having been part of the organizing team for the very first one, I was impressed   to see what it had developed into. When I came to get my badge the energetic and enthusiastic volunteer asked me if I was “pumped”, but I was not pumped (as far as my understanding of what that meant) so I politely replied that I was probably as pumped as I was ever going to be.

Inside was packed and at one point a fascist looking guy pushed me and told me to step aside, just as I was getting ready to put up a fight and stand my ground I noticed the crown prince of Denmark strolling by. So, I left him with a warning and let him off the hook for this time (maybe if I had been some more pumped…also I suspect that all of this played out as a blank stare from the point of view of the body guard)

At the exhibition floor I had the good fortune of chatting with a few McKinsey consultants at their booth. The couches were exquisite and so would the coffee have been if they had offered me some. If there is one thing McKinsey can do it is talk and do research and currently they do a lot of talk and research on Artificial Intelligence (AI). I was lucky to get my hands on some of their reports that detail their look on Artificial Intelligence in general and AI in the Nordics in particular. 

The main story line is the same one that you hear everywhere: AI is upon us and it promises great potential if not a complete transformation of the world as we know it. There are however a few conclusions that we should dive into a little bit more. 

The wonders of AI

In terms of investment in AI, 2/3 of businesses allocate 3% or less of investments in AI and only 10% allocate 10% or more. If you were reading the tech news you would be forgiven for thinking that 90% of companies were investing a 100% or more in AI. So, this observation alone is interesting. There is not a lot of actual investment going towards AI for the vast majority of companies. When you ask senior management and boards there is a bit of a waiting game, where they look more towards competitors moves than to the actual potential of AI. 

The status of adoption is that in the Nordics 30% (compared to 21% globally) of companies have embedded at least one AI technology across their business. This could be taken to mean that the Nordics were ahead of the curve compared to the global market. It could also be due to the Nordics having a higher general level of digitalization. 

These things taken together it seems that AI as a technology is still in Innovators/the early adopter category in the diffusion of innovation theory developed by Everett M. Rogers. Rogers developed a framework and body of research that has been shown across multiple industries and technologies that show the patterns of adopting innovations of any type. AI is one such type of innovation, just like the Iowa farmers’ adoption of 2.4-D weed spray that was Rogers initial focus of investigation more than 50 years ago. The research showed that the adoption took the form of a bell curve.

 

Figure 1. Diffusion of innovations, credit: Wikimedia commons

 

The fact that companies are waiting for competitors to use AI also clearly indicates that we are in the early adopter or early majority category, as this is typical behavior for the early adopters. Whereas innovators will go with anything as long as it is  new, early adopters are more picky. Early Majority are primarily looking at what the competition is doing in order to copy them. 

If we look at figure 2 we can see that companies that have adopted AI today are vastly more profitable. The logic seems to be straight forward: there is a huge potential for AI to make companies more profitable.

 

 

Figure 2. AI adoption and profit margins (source: McKinsey Global Institute ) 

While this is indeed a tempting conclusion, we have to be cautious. Keep in mind that the companies adopting AI may just be more technologically proficient. The AI adoption could be confounded with adopter category and technology utilization in general. It could just mean that companies more open to innovation of any kind are on average more profitable than those who are not. It is well known that especially early adopters are more profitable than other adopter categories. 

To put it another way: adopting AI may result in you becoming more profitable, but is not certain that AI is the reason. What McKinsey doesn’t tell us, but I expect them to know full well, is that the reverse is also true. Investing in AI may actually set  you up for failure. 

AI adoption and adopter category

The issue here is that it may not be AI that is making the companies profitable, it may rather be their adopter category. The adopter category is related to their company culture. A company culture that is friendly to new technologies will behave as an early adopter and  monitor the market and selectively choose solutions that they think will give them an advantage. This is what they do with any type of technology, not just AI. But we also have to remember that the reason they are successful is exactly because of their company culture and the fact that they are used to trying out new solutions.

They know that when they invest in something new you don’t just press install, next, next, finish and the money starts flowing. They know that new technologies are rough around the edges and there is going to be a lot of stop and start and two steps forward and one step back. They are driven by a belief that they will fix it somehow. More importantly, they have a sufficient amount of people with a “can-do attitude” that are not afraid to leave their comfort zone (see figure 3)

 

 

Figure 3. where magic happens

 Now, compare this with organizations that have more people of a “not-invented-here attitude”. Their company culture leads them to the late majority and laggard categories. For this type of organization, innovations are something to be shunned, they know what they are doing and consider it a significant risk to do anything differently. Their infrastructure is not geared towards making experimental and novel technologies work. It is geared towards efficiently and making well known technology work in a predictable manner.

Let’s do a thought experiment about how this will play out: Karma Container, a medium sized shipping company, decides to send Fred, an inspired employee, to TechBBQ . They still have mission critical applications running on the mainframe and Windows NT servers (because Linux or MacOS are not in use anywhere) and upgrades are a major concern that has the CIO biting his nails every time. Fred comes back from the conference energized. He spoke to the same McKinsey consultants and read the same reports that I did. He pitches to his CIO that they should invest in AI because the numbers clearly indicate that it would increase the company’s profitability. The last time they invested in any new technology was to transfer their telephones to IP telephony and implement help desk technology. The CIO says ok, and they decide to try to adopt a chatbot to integrate with their helpdesk and website.

So, with a budget and a formal project established Fred starts. They wonder who in the organization would actually implement it. They go to the database administrator, who looks at them as if they were suddenly speaking a different language. He has no idea. They go to the .net developer who fails to appreciate how that could in any way involve him. They then go to the system administrators, who quickly show them to the door on account of a purported acute security event. They don’t get back to the project team either.

Remember that at this point they haven’t even started to figure out who would maintain, patch and upgrade the system or who would be responsible when it behaves strangely or who would support it. Fred quickly gives up and returns to his job of managing Remedy tickets.

 

Beware of AI

 The purpose of this thought experiment (vaguely based on real life experience even though the names and details have been changed) is that even if AI does have much to offer in terms of profitability and efficiency it is not a realistic choice for most companies at this point. I would even go so far as to say that all AI should be avoided by most companies unless they have a track record and company culture that would indicate they could make it work.

Most AI solutions are not mature enough, that is easy enough to use,  and more importantly the value proposition is speculative. If an organization is not geared towards implementing experimental technologies, they are wasting time, money and effort on trying. This is why most companies are better off waiting. This is similar to websites in the 1990ies. They were not for everyone, but today anyone can click a few times and create a beautiful site in WordPress or other CMS. Once we have the equivalent of a wordpress for AI, that is when most companies should invest.

Diffusion of innovations just takes time it cannot and should not be forced. The current AI hype is also a result of innovators and early adopters being more loud and opinion forming than most companies. Most companies are better off waiting for the dust to settle and more mature and comprehensive solutions to appear

 

Posted on

AI, Traffic Tickets and Stochastic Liberty

Recently I received a notification from Green Mobility the electric car ride-share company I am using some times. I have decided not to own a car any longer and experiment with other mobility options, not that I care about the climate, it’s just, well, because. I like these cars a lot, I like their speed and acceleration and that you can just drop them off and not think about them ever again. Apparently I seem to have enjoyed the speed and acceleration a little too much, since the notification said the police claimed that I (allegedly) was speeding on one of my trips. For a very short period of time I toyed with the “It-wasn’t-me” approach, but quickly decided against that since technology was quite obviously not on my side here. Then I directed my disappointment at not receiving complete mobility immunity along with all the other perks of not owning my car against the company charging me an extra fee on top of the ticket, a so called administration fee. But that was a minor fee anyway. Then I decided to rant against the poor support person based on the fact that they called it a parking ticket in their notification and that I obviously wasn’t parking according to the photo. Although in my heart I did realize that this was not going anywhere.

I believe this is a familiar feeling to any of my fellow motorist: the letter in the mail displaying your innocent face at the wheels of your car and a registered speed higher than allowed along with payment details of the ticket you received for the violation. It is interesting to observe the anger we feel and the unmistakable sense that this is deeply unfair even though it is obviously not. The fine is often the result of an automated speed camera that doesn’t even follow normal working hours or lunch breaks (an initial reason for it being unfair). A wide suite of mobility products like GPS systems and Waze keeps track of these speed cameras in real time. Some people follow and report this with something approaching religious zeal. But what is the problem here? People know or should know the speed limit and know you will get a ticket if you are caught. The operative part of this sentence seems to be the “if you are caught” part. More about that in a minute.

The Technology Optimisation Paradox

Last year I was working with for the City of New York to pilot a system that would use artificial intelligence to detect different things in traffic. Like most innovation efforts in a city context it was not funded beyond the hours we could put into it, so we needed to get people excited and find a sponsor to take this solution we were working on further. Different suggestions about what we should focus on came up. One of them was that we should use the system to detect traffic violations and automatically fine car owners based on the license plate.

This is completely feasible, I have received tickets myself based on my license plates, so I gathered that the technology would be a minor issue. We could then roll it out on all the approximately 3000 traffic cameras that are already in the city. Imagine how much revenue that could bring in. It could probably sponsor a couple of new parks or sports grounds or even a proper basket ball team for New York. At the same time it would improve traffic flow because less people would double park and park in bus lanes etc. When you look at it, it seems like a clear win-win solution. We could improve traffic for all New Yorkers, build new parks and have a team in the NBA Play Offs (eventually). We felt pretty confident.

This is where things got complicated. We quickly realized that this was indeed not a pitch that would energize anyone, at least not in way way that was beneficial to the project. Even though people are getting tickets today and do not suggest seriously that they should not, the idea of OPTIMIZING this function in the city seemed completely off. This is a general phenomenon in technological solutions, I call this the “Technology Optimization Paradox”: when optimizing a function, which is deemed good and relevant leads to resistance at a certain optimization threshold. If the function is good and valuable there should be no logical reason why doing it better should be worse, but this is sometimes how people feel. This is the technology optimization paradox. It is often seen in the area of law enforcement. We don’t want surveillance even though that would greatly increase the fight against terrorism. We like the function of the FBI that lead to arrests and exposure of terrorist plots but we don’t want to open our phones to pervasive eaves dropping.

Stochastic Liberty

This is where we get back to the “If you are caught” part. Everyone agrees that it is fair that you are punished for a crime if you are caught. The emphasis here is on the “if”. When we use technology like AI we get very very close to substituting the “if” with a “when”. This is what we feel is unfair. It is as though we have an intuitive expectation that we should have a fair chance of getting away with something. This is what I call the right to stochastic liberty: The right for the individual to have events to be un-deterministic. Especially adversary events. We want to have the liberty to have a chance to get away with an infringement. This is the issue many people have with AI when it is used for certain types of tasks, specifically tasks that have an optimization paradox. It takes away the stochastic liberty, it takes away the chance element.

Let us look at some other examples. When we do blood work, do we want AI to automatically tell us about all our hereditary diseases, so the doctor can tell us that we need to eat more fiber and stop smoking? No sir,  we quietly assert our right to stochastic liberty and the idea that maybe we will be the 1% who lives to be 90 fuelled on a diet of sausages, fries and milkshake even though half our family died of heart attacks before they turned 40.  But do we want AI to detect a disease that we have suspicion that we might have? Yes!

Do we want AI to automatically detect when we have put too many deductions on our tax return? No way, we want our stochastic liberty. Somebody in the tax department must sit sweating and justify why regular citizens tax returns are being looked through. At most we can accept the occasional spot test (like the rare traffic police officer, who also has to take a break and get lunch and check the latest sport results, thats fair). But do we want AI to help us find systematic money laundering and tax-evation schemes: hell yeah!

Fear of the AI God

Our fear of AI is that it would become this perfect god that would actually enforce all the ideals and ethics that we agree on (more or less). We don’t want our AI to take away our basic human right of stochastic liberty.

This is a lesson you don’t have to explain to politicians who ultimately run the city and decide what gets funded and what not. They know that unhappy people getting too many traffic tickets that they think are unfair, will not vote for them. This is what some AI developers and technocrats do not appreciate when they talk about how we can use AI to make the city a better place. The city is a real place where technology makes real impacts on real people and the dynamics of technology solutions exceed those of the system in isolation. This is a learning point for all technology innovation involving AI: there are certain human preferences and political realities that impose the same limits on the AI solution as the type of algorithm, IOPS and CPU usage.

 

Posted on

The Challenges of Implementing AI Solutions – A Personal Journey

For about a decade I have been involved in various system development efforts that involved Artificial Intelligence. They have all been challenging but in different ways. Today AI is rightfully considered a game changer in many industries and areas of society, but it makes sense to reflect on the challenges I have encountered in order to asses the viability of AI solutions.

10 years of AI

About 10 years ago I designed my first AI solution, or Machine Learning as we typically called it back then. I was working in the retail industry at that time and was trying to find the optimal way of targeting individual customers with the right offers at the right time. Lots of thought went into it and I worked with an awesome University Professor (Rune Møller Jensen) to identify and design the best algorithm for our problem. This was challenging but not completely impossible. This was before TensorFlow or any other comprehensive ML libraries were developed. Never the less everything died due to protracted discussions about how to implement our design in SQL (which of course is not possible: how do you do a K-means clustering algorithm in SQL), since that was the only language known to the BI team responsible for the solution.

Fast forward a few years I find myself in the financial services industry trying to build models to identify potential financial crime. Financial crime has a pattern and this time the developers had an adequate language to implement AI and were open to use the newest technologies such as Spark and Hadoop. We were able to generate quite a few possible ideas and POCs but everything again petered out. This time the challenge was the regulatory wall or rather various more or less defined regulatory concerns. Again the cultural and organizational forces against the solution were too big to actually generate a viable product (although somehow we did manage to win a big data prize)

Even more fast-forward until today. Being responsible for designing data services for the City of New York the forces I encountered earlier in my career are still there, but the tides are turning and more people know about AI and take courses preparing them for how it works. Now I can actually design solutions with AI that will get implemented without serious internal forces working against it. But the next obstacle is already waiting and this time it is something particular to government and not present in the private industry. When you work for a company it is usually straightforward to define what counts as good, that is, something you want more of like say, money. In the retail sector, at the end of the day all they cared about were sales. In the Financial Services sector it was detecting financial crime. In the government sector that is not as straight forward.

What drives government AI adoption?

Sure, local, state and federal government will always say that they want to save money. But really the force driving everything in government is something else. What drives government is public perception, since that is what gets officials elected and elected officials define the path, appoint directors who hire the people who will ultimately decide what initiatives get implemented. Public perception is only partially defined by efficiency and monetary results. There are other factors that interfere with success such as equity, fairness, transparency etc.

Let me give some examples to explain. One project I am working on has to do with benefits eligibility. Initially City Hall wanted to pass legislation that would automatically sign up residents for benefits. However, after intervention by experts this was changed to doing a study first. The problem is that certain benefits interfere with other benefits and signing you up for something that you are eligible for may affect you negatively because you could loose another benefit.

While this is not exactly artificial intelligence it is still an algorithm that displays the same types of structural characteristics: the algorithm magically makes your life better. Even if we could make the algorithm count the maximum benefit of all available benefits and sign the resident up for the optimal combination, we still would not necessarily thrill everyone. Since benefits are complex it might be that some combination will give you more in the long term rather than the short term. What then if the resident prefers something in the short term? What if the system fails and a family gets evicted and has to live in a shelter because the system failed to detect eligibility due to bad master data?

When I was in the retail industry that would amount to a vegetarian getting an offer for steaks. Not optimal but also not critical if we could just sell 10 more steaks. In the financial services industry it would amount to a minor Mexican drug lord successfully laundering a few hundred thousand dollars. Again, this is not great but also not a critical issue. In government a family being thrown out on the street is a story that could be picked up by the media to show how bad the administration is. Even if homelessness drops 30% it could be the difference between reelection and loss of power.

What does a success look like?

So, the reward structures are crucial to understand what will drive AI adoption. Currently I am not optimistic about using AI in the City. Other recent legislation has mandated algorithmic transparency. The source code for every algorithm that affects decisions concerning citizens needs to be open to the public. While this makes sense from a public perception perspective it does not from a technical one. Contrary to popular belief I don’t think AI will catch on in the government any time soon. I think this can be generalized to any sector where the reward function is multi-dimensional, that is, where success cannot be measured by just one measure.

Posted on

Do Product Managers Need to have Programming Experience?

Do you need to be able to program to be a good product manager? Opinions differ widely here.

Full disclosure: I have very little if any meaningful command of any programming language. If you feel you need to be able to program in order to have an informed opinion, you have already answered the question yourself and can safely skip this and read on.

So, just to get my answer out of the way: “no”

I would say just as you don’t need to know how to lay bricks in order to be an architect or be a veterinarian in order to ride a horse.

When I hear people, who answer “yes” to the question I always want to counter: is it necessary to know anything about humans in order to build tech products for humans? Very few, if any, make products that do not crucially depend on and interact with humans, but it has always been curious to me why that part of the equation is always assumed to be trivial and not requiring any sort of experience or education.

This is even more puzzling when you consider that the prevalent cause of product failure seems to be the human part of it. Let me just mention three examples.

Remember google glass? That was a brilliant technology, but a failed product due to a lack of understanding of what normal humans think is creepy. I wrote about this back in 2014 and observed

A product has to exist in an environment beyond its immediate users. Analysis of this environment and the humans that live in it could have revealed the emotional reactions.

 

Remember autonomous vehicles? Perfect technology, but unfortunately not necessarily considered as such by the humans that run the imperceptible risk of being killed by it and who lives with the result of the actions of the AI, which will eventually be traced back to humans somewhere. This is something I touched on in a recent blog post.

We would still have to instill the heuristics and the tradeoffs in the AI, which then leads back to who programs the AI. This means that suddenly we will have technology corporations and programmers making key moral decisions in the wild. They will be the intelligence inside the Artificial Intelligence.

 

Similarly for features of products like the number of choices you have. You might assume that the more choice the more value to the product, but keep in mind that if the product is used by humans you have to think about the constraints humans bring:

In general the value of an extra choice increases sharply in the beginning and then quickly drops off. Given the choice of apples, oranges, pears, carrots and bananas are great, but when you can also choose between 3 different sorts of each the value of being offered yet another type of apple may even be negative. The reason for this phenomenon has to do with the limits of human consciousness.

 

The root cause of product failure is typically not technical but human, so rather than asking a product manager for his command of programming languages maybe do a check on where he or she falls on the autism spectrum. Maybe ask whether he has ever studied anything related to the human factors like psychology, anthropology, sociology or similar topics that would allow him to make products that work well for humans.

 

This post is based on my response on Quora to the question: “Is it necessary for a product manager to know programming language?”

 

Posted on

AI is Easy – Life is Hard

Artificial Intelligence is easy. Life is hard. This simple insight should collectively caution our expectations. When we look at Artificial Intelligence and the amazing results it has already produced, it is clear that it has not been easy. The most iconic victories are as follows.

  • Deep Blue beats Kasparov
  • Watson beats champions Brad Rutter and Ken Jennings in jeopardy
  • Google beats world champion in Go

Today Autonomous Vehicles (AV) manage to stay on the road and go where they are expected to. Put on top of this multiple implementations of face recognition, speech recognition, translation etc. What more could we want to show us that it is just a matter of time before AI becomes truly human in its ability? Does this not reflect the diversity of human intelligence and how it has been truly mastered by technology?

Actually, no, I don’t think so. From a superficial point of view it could look like it, but deep down all these problems are if not easy, then hard in an easy way in the sense that there is a clear path to solving them.

 

AI Is Easy 

The one thing that holds true for all of these applications is that the goals are very clear. Chess, Jeopardy and Go: you either win or you don’t. Facial, speech and any other kind of recognition: you recognize something or you don’t. Driving an autonomous vehicle: It either drives acceptably according to the traffic rules or it doesn’t. If only human life were so simple.

Did you know when you were born what you wanted to work with? Did you know the precise attributes of the man/woman you were looking for? Did you ever change your mind? Did you ever want to do two or more mutually exclusive things (like eating cake for breakfast and live a healthy life)?

Humans are so used to constantly evaluate trade offs, with unclear and frequently changing goals that we don’t even think about it.

 

An AI Thought Experiment 

Let me reframe this in the shape of a tangible existing AI problem: Autonomous Vehicles (AV). Now that they are very good or even perfect at always staying within the traffic rules, how do they behave when conditions are not as clear? Or even in situations where the rules might be conflicting?

Here is a thought experiment: the self-driving car is driving on a sunny spring afternoon through the streets of New York. It is a good day and it is able to keep a good pace. On its right is a sidewalk with a lot of pedestrians (as is common in New York), on its left is a traffic lane going the opposite direction as they do on two-way streets (which are more rare but not altogether absent). Now suddenly a child runs out into the road in front of the car and it is impossible for it to brake in time. The autonomous vehicle needs to make a choice. It either runs over the child makes an evasive maneuver to the right hitting pedestrians or the left hitting cars going the other direction?

How do we prepare the AI to make that decision? Now, the goals are not so clear as in a jeopardy game. Is it more important not to endanger children? Let’s just say for the sake of argument this was the key moral heuristic. The AI would then have to calculate how many children were on the sidewalk and in a given car on the opposite side of the road. It may kill two children on the sidewalk or in another car. What if there were two children in the Autonomous Vehicle itself? Does the age factor in to the decision? Is it better to kill old people than younger? What about medical conditions? Would it not be better to hit a terminal cancer patient than a healthy young mother?

The point of this thought experiment is just to highlight that even if the AI could make an optimal decision it is not simple what optimal means. It may indeed differ across people, that is, regular human beings who would be judging it. There are hundreds of thousands of similar situations where there just by definition is no one right solution, and consequently no clear goal for the AI to optimize towards. What if we had an AI as the next president? Would we trust it to make the right decisions in all cases? Probably not, politics is about sentiment, subjectivity and hard solutions. Would we entrust an AI that would be able to go through all previous court cases, statistics, political objectives to make fair rulings and sentencing? No way, although it probably could.

 

Inside the AI Is the Programmer 

As can be seen from this the intelligence in an AI must be explained by another intelligence. We would still have to instill the heuristics and the tradeoffs in the AI, which then leads back to who programs the AI. This means that suddenly we will have technology corporations and programmers making key moral decisions in the wild. They will be the intelligence inside the Artificial Intelligence.

In many ways this is already the case. A more peaceful case in point is online dating: a programmer has essentially decided who should find love and who shouldn’t through the matching algorithm and the input used. Inside the AI is the programmer making decisions no one ever agreed they should. Real Artificial Intelligence is as elusive as ever; no matter how many resources we throw at it. Life will throw us the same problems as it always has and at the end of the day the intelligence will be human anyway.

Posted on

AI and the City

Artificial Intelligence is currently being touted as solution to most problems. Most if not all energy is put into conjuring up new and even more exotic machine learning models and ways of optimizing these. However, the primary boundary for AI is currently not technical as it used to be. It is ecological. Here I am not thinking about the developer ecosystem, but the ecosystem of humans that have to live with the consequences of AI and interact with machines and systems driven by it. While AI lends itself beautifully to the concept of smart cities this is also one of the avenues where this will most clearly play out because the humans that stand to benefit and potentially suffer from the consequences of AI are also voters. Voters vote for politicians and politicians decide to fund AI for smart cities.

How Smart is AI In A Smart City Context?

At a recent conference I had an interesting discussion where we were talking about what AI could be used for. Someone suggested that Machine Learning and AI could be used for smart cities. Working for a city and having worked with AI for a number of years, my question was “for what?” One suggestion was regulating traffic.

So, let us think through this. In New York City we have on occasion a lot of traffic. Let us say that we are able to construct a machine learning system that could indeed optimize traffic flow through the city. This will not be simple or easy, but not outside the realm of the possible. Let us say that all intersections are connected to this central AI algorithm that provides the city as a whole with optimal traffic conditions. The algorithm works on sensor input that counts the number of cars at different intersections based on existing cameras. Probably this will not mean that traffic always flows perfectly but certainly on average does better.

Now imagine during one of these congestions a fire erupts in downtown Manhattan and fire trucks are delayed in traffic due to congestion. 50 people die. The media then finds out that the traffic lights are controlled by an artificial intelligence algorithm. They ask the commissioner of transportation why 50 people had to die because of the algorithm. This is not a completely fair question but media have been known to ask such questions. He tries to explain that the algorithm optimizes the overall flow of traffic. The media are skeptical and ask him to explain how it works. This is where it gets complicated. Since this in part is a deep learning algorithm no one can really tell how it works or why there was congestion delaying the fire trucks at that particular time. The outrage is palpable and headlines read “City has surrendered to deadly AI” and “Incomprehensible algorithm leads to incomprehensible fatalities”

Contrast this to a simple algorithm that is based on clear and simple rules that are not as effective overall but work along the lines of 30 seconds one way 30 seconds another way. Who would blame the commissioner of transportation for congestion in that case?

Politics And Chaos

Media aside there could be other limiting factors. Let us stay with our idea of an AI system controlling the traffic lights in New York City. Let us further assume that the AI system gets a continuous input about traffic flow in the city. Based on this feed it can adapt the signals to optimize the flow. This is great but due to the fact that we have now coupled the system with thousands of feedback loops it enters into the realm of complex or chaotic systems and will start to exhibit properties that are associated with that kind of systems. Typical examples of such properties are: erratic behavior, path dependency, and limited possibility for prediction.

Massively scalable AI cannot counteract these effects easily and even if we could, the true system dynamics would not be known until the system goes live. We would not know how many cars would be running red lights or speed up/slow down compared to today. Possibly the system could be trimmed and be made to behave, but then we basic politics. Which responsible leader would want to experiment with a city of more than 10 million peoples daily lives. Who would want to face these people and explain to them that the reason they are late for work or for their son’s basket ball game is trimming an AI algorithm?

The Limits Of AI

So, the limits to AI may not be primarily of a technical nature. They may have just as much to do with how the world behaves and what other non-data-scientist-humans will accept. Even if it is better to loose 50 people in a fire in Manhattan once every 10th year and reducing the number of traffic deaths by 100 every year, the stories written are about the one tragic event not about the general trend. Voters remember the big media stories and will never notice a smaller trend. Consequently, regardless of the technical utility and precision of AI, there will be cases where the human factor will constrain the solutions more than any code or infrastructure.

Based on this thought experiment I think the most important limits to adoption of AI solutions at city scale are the following

  • Unclear benefits – what are the benefits of leveraging AI for smart cities? We can surely think up a few use-cases but it is harder than you think. Traffic was one but even here the benefits can be elusive.
  • Algorithmic transparency – if we are ready to let our lives be dominated by AI in any important area citizens who vote will want to understand precisely how the algorithms work. Many classes of AI algorithms are incomprehensible in their nature and constantly changing. How can we prove that no one tampered with them in order to gain an unfair advantage? Real people who are late for work or are denied bail will want to know that and some times Department Of Investigation will want to know as well.
  • Accountability – whatever an algorithm is doing, people will want to hold a person accountable for it if something goes wrong. Who is accountable for malfunctioning AI? Or even well functioning AI with unwanted side effects? The buck stops with the responsible on the top, the elected or appointed official.
  • Unacceptable implementation costs – real world AI in a city context can rarely be adequately tested in advance as we are used to for enterprise applications. Implementing and trimming a real world system may have too many adversarial effects before it starts to be beneficial. No matter how much we prepare we can never know exactly how the human part of the system will behave at scale until we release it in the wild.

Artificial Intelligence is a great technological opportunity for cities but we have to develop the human side of AI in order to arrive something that is truly beneficial at scale.