Through a project called We Count, my team and I pulled together an amazing community focused on ensuring that people, who are different from what is deemed average, are not excluded, ignored, misunderstood, mistaken as threats; and their data is not abused, or misused by emerging systems. We are specifically concerned with decision systems, artificial intelligence, and data systems that rely on large amounts of data to arrive at predictions, a simplified truth for the majority, or an efficient judgement that ignores exceptions. As part of this project, we published a newsletter, and I wrote a “Director’s Message” for each newsletter. I am sharing these messages here as a series of blogs.

June 2022

You have likely heard a great deal of buzz about Meta and the emerging Metaverse. This is being promoted as the next disruptive innovation. I was invited to present a keynote in the Metaverse. I wanted to share a transcript of what I said. I would love your thoughts and feedback.

“Thank you for this opportunity to speak in this emerging domain.

I have a confession to make.

The organizers asked that I share news of the event with my community and invite some bright, creative young students to help in building this world. I said I would. But on Tuesday, as I was preparing to send the invitation to our community list, I hesitated. You see, my community, the students, my team members, the many partners and colleagues, include many community members reliant on alternative access systems. I would have loved to have shared your work with the community I am part of. However, I tested the environment with a screen reader, a screen magnifier, and a scanning keyboard, and I faced barriers after barriers. I tried to ascertain whether there would be any forms of interpretation and how they would be accessed. I considered the bandwidth, the equipment that would be needed and realized that quite a few community members don’t have it. So, I didn’t send out the invitation, because I knew that only certain members in the community could participate, and that wasn’t fair. To be clear, the community members that would be left out would likely tell me to send it, they would likely say “don’t let me hold you back.” But they shouldn’t be left behind or left back. In fact, I believe that not barrelling forward without them, but putting them in front, is how we will truly, meaningfully go forward.

I do want to share the excitement, the excitement of something new, something emergent. I’m old enough and I’ve been in this field enough that I have participated in this heady phase of creativity more than once, with personal computers, with the Web, with social media, with smart phones, with artificial intelligence, with sharing platforms, with the cloud, with disintermediated value exchange, with VR, AR and XR.

There is this thrilling moment in time that I love. The moment of possibility, imagination, and unfettered choices. I always imagine that we will do something truly innovative. The moment is fleeting, before the new terrain is claimed by many forces with predetermined interests, serving their established agendas. Before the conventions set in and imagination is blinkered by what is, rather than what could be. Well before the ideals are sullied, and unbounded potential is shackled with conditions. I know that is the moment when we have a chance to do something world changing. Unfortunately, we keep falling short. We have never stayed the course to true, deep, world altering innovation. There has been a disheartening failure of the imagination by the innovators and disruptors. What is sold to us as innovation is simply a new costume on old ideas and mindsets.

So, what do I mean by true innovation? Not just a new fashion, a new trend, or a new way of expressing old things. I don’t mean shiny new objects that can do new tricks, a fad that can be monetized to make ridiculous profit. I mean a deep and lasting change that is more than a toy for the elite that can afford it, more than a convenience, more than a new addiction or pastime to compete for our attention.

I’m not saying this to spoil the fun. I like fun, I love to play, I especially like playing with my friends, with my loved ones. But that means that we need to create a place where they can participate and enjoy playing.

I fear that the innovation field is not really innovating. We are simply creating variations and manifestations of the innovative ideas and mindsets of the 1800’s, and replicating age-old ethical flaws.

We are still convinced that valid humans are normal or average, and if you aren’t we will school you, train you, fix you to be normal. Quantified statistics is a quest to find the average or the norm. Our latest trends are all the devices and apps that give us metrics to track that we are achieving normalcy, or to find our way back to what we determine is normalcy. There is no average human, we are all diverse. Our diversity is our strength. What we should be seeking is the range of what we call the deviation, the amazing spectrum of human differences.

We know about the need for diversification from the life sciences, but we think the biggest lesson is survival of the fittest, rather than the evolutionary choices that emerge when we relax competitive selection. We think fitness is might, cut-throat competition, and power. We forget it is adaptability, and symbiosis or collaboration. I worry about all the potentials for innovation we pass up when we fail to collaborate, passing up the alchemy that occurs when diverse great ideas are combined.

We are still in the thralls of the 80/20 rule, but we are looking at it backwards if we want innovation. Rather than designing for the 80 percent who take 20 percent of the effort, we should be designing with the 20 percent who occupy the unexplored and difficult terrain that will take 80 percent of the effort, but costs less in the long run and gives us dynamic resilience. Our market research shouldn’t try to find the largest customer base, it should find the people who are not served.

Most tragically we think we need to categorize, sort and label humans, fit them into boxes. Even the diversity, equity and inclusion efforts think this is a good thing to achieve equality. This gives us a score for the balance of power. Equity is more than a balancing of power. We need to rethink how we give power and how we use it.

So what’s my bright idea?

To find genuine innovation, we need to innovate the process of innovation.

First who we innovate with, who helps us design the process of innovation. Innovation comes from challenges. The people that face the greatest challenges will bring the greatest impetus for innovation. They need to frame the problems.

I’m not religious but I think that the many forms of the morality tale of the angel, saint, prophet or god as a beggar in disguise, and the reward to those that treat them with respect and care is apt for innovation. The people that hold the formula and impetus for true innovation are the people we have excluded and devalued. It isn’t the complacent, comfortable, powerful, popular or wealthy, no matter how much they claim the title of innovator.

We need to go even deeper. We need to innovate how we make decisions (not just majority rules, or consensus arrived through influence), how we plan (not linear logic models because life is not linear; it isn’t predictive analytics — predictive analytics finds the best plan for the past not the unpredictable future). We need to innovate how we determine evidence and truth (not statistical significance that only finds truth for the mythical average and is inaccurate or wrong for the margins). We need to innovate how we protect our innovation, by that I don’t mean intellectual property protection that secures ownership for profit. I mean building in the safeguards so it cannot be abused, misused or weaponized, so we make it clear that with freedom comes responsibility.

For the accessibility community we need to innovate how we do accessibility. It isn’t a check list for a formula that denies the essence of disability — human difference. Equity is not a ritual we perform without any deep reflection.

With my community I have captured a draft roadmap for genuine innovation. We call it the three dimensions of inclusive design.

The first dimension is recognizing that we are all different and we are the experts in our difference. This vast range of differences needs to be addressed in an integrated not segregated way.

The second dimension is that we need an inclusive process, we need to bring the largest diversity of perspectives to the process, it needs to be the individuals with the greatest challenges that help design the process and frame the challenges. We need to continuously ask who is missing, who have we left behind with this innovation.

The third dimension is recognizing that we live in a complex adaptive system. We need to seek benefit for all. Nothing is done in isolation. Everything is entangled and variable. Problems are not monocausal, the path out of problems is not linear or formulaic. There is no fix, solution or success, it is a continuous process. But it is the people at the margins that are most familiar with the risks and opportunities.

We apply these dimensions in what we call our virtuous tornado, unlike the design thinking squiggle which iterates to a winning solution, we co-create an adaptable system that stretches to encompass more and more of the missed needs, by continuously asking “who are we missing?” We appreciate the imperfect, impermanent and incomplete and know that mistakes and failures yield the greatest learning.

I trust and hope that this amazing community here can find true innovation. “

What are your thoughts about the Metaverse? Will it lead us to true innovation or does it reinforce old patterns?

All my best,

Jutta

**Please note this work is licensed under an Attribution-NonCommercial 4.0 International License**

Through a project called We Count, my team and I pulled together an amazing community focused on ensuring that people, who are different from what is deemed average, are not excluded, ignored, misunderstood, mistaken as threats; and their data is not abused, or misused by emerging systems. We are specifically concerned with decision systems, artificial intelligence, and data systems that rely on large amounts of data to arrive at predictions, a simplified truth for the majority, or an efficient judgement that ignores exceptions. As part of this project, we published a newsletter, and I wrote a “Director’s Message” for each newsletter. I am sharing these messages here as a series of blogs.

May 2022

Friends and Colleagues,

Accuracy, consistency, and efficiency seem like good qualities. Who wouldn’t want these qualities in the running of a business, a school, a healthcare system, a city? If we can automate these qualities, all the better. Our businesses, schools, healthcare systems and cities will run like “well-oiled machines.” Fewer hassles, far less cost, no need to decide or negotiate at every interaction. The procedures and services will simply run accurately, consistently, and efficiently. This is the promise of machine learning, artificial intelligence, smarts, and automation that many administrators are hoping to achieve.

I see several problems.

The first problem is that people are very diverse, often unpredictable, and their lives are unruly. Anyone not taken into account in the design of the “intelligent” machine will likely become a casualty of the machine or someone that “gums up the works.” Disability by its very nature means you don’t fit the machine. Only some people with disabilities have machines that another machine can recognize, like a wheelchair. Most disabilities are outside the range of prediction.

The second problem is that the process is taken for granted and no longer questioned. It becomes just the way things are done, something that is not negotiable. This is especially the case when there are too many “sunken costs” or investments in making it work. This means that we don’t question the value of what we are automating. Do all students need to take standardized tests? Do we need to give all that information about ourselves to get a simple service? Do we need all these security procedures and access restrictions? What we have mechanized ossifies and becomes part of the infrastructure. We build more systems on top of this infrastructure, and it becomes harder and harder to change.

Lastly, we lose serendipity, flukes, the unexpected, and novelty. With the automation of decisions, we reduce the choices we are offered at each interaction. This means that we are collectively less adaptable. We can’t respond as well when our world changes. Often, the only way things change is when something sufficiently disruptive happens that the whole machine breaks down.

All machines, even highly intelligent machines, have difficulty dealing with the unexpected person and the unruly parts of our lives. At the moment, smart systems can only view these as a threat or a mistake. We need to think carefully about the cost of those desirable qualities accuracy, efficiency and consistency before we hand over our human processes to machines.

All my best,

Jutta

**Please note this work is licensed under an Attribution-NonCommercial 4.0 International License**

Through a project called We Count, my team and I pulled together an amazing community focused on ensuring that people, who are different from what is deemed average, are not excluded, ignored, misunderstood, mistaken as threats; and their data is not abused, or misused by emerging systems. We are specifically concerned with decision systems, artificial intelligence, and data systems that rely on large amounts of data to arrive at predictions, a simplified truth for the majority, or an efficient judgement that ignores exceptions. As part of this project, we published a newsletter, and I wrote a “Director’s Message” for each newsletter. I am sharing these messages here as a series of blogs.

I often say that our goal as inclusive designers is to design inclusive systems. Our We Count project talks about creating an inclusive data ecosystem. The next thing people naturally ask is “can you point to the best examples of completed inclusive designs?” An associated question is often, which countries are the best at inclusive design.

This comes from people’s experiences of universal design and accessibility. Universal design arose from architecture. Architecture is completed and relatively static. You can point to it and trust that it will “stay put.” The accessibility field has adopted a notion of “full accessibility” which has a fixed set of testable criteria that are used to certify that something is accessible. You can point to certified examples.

Inclusive design differs in some fundamental ways. The first is that inclusive design is a process, not a thing or a building. The process involves a system such as an education system, a place of work, a healthcare system, a community, a market, an economy, or even a system of government. These systems are complex and adaptive. They change in unpredictable ways; they do not stay put. There are many dimensions, processes, influences, roles, and mindsets that are part of a system. Inclusive designers intervene in systems to help prepare the system to be more adaptive, support greater diversity, and become more dynamically resilient. Most importantly inclusive designers equip people to take on the role of designing their evolving system more inclusively.

This process can’t be engineered, it is grown. Attempts at rigid control result in systems that are brittle, that won’t survive the next change. Inclusive approaches often begin with small successes, but they must consider the entire system. Take the school example, if you co-design an innovative inclusively designed teaching approach with a student who can’t use the standard approach, it won’t survive if the teacher doesn’t fully support it; the teacher will have problems if the principal doesn’t support it; the principal will have issues if the school board doesn’t support it; and the school board will have difficulty if the Ministry of Education doesn’t understand and value the change. The parents, the educational assistant, the technology suppliers, the student’s classmates also play a role. At each point in the nested system there will be friction if the entire system is not prepared. The people at the friction point will bear the load.

So how do you know you are successful? Our goal is systems with the participation and belonging of the greatest range of human difference. This is what supports a system that is most adaptive. Our most reliable indicator of progress is that people who have been marginalized or excluded by the system are faring better. The only way to be sure of this is to ensure they are full participants in the design decisions from the start. It is not primarily about changes in the power structure, but in the entire process of how decisions are made, and power is used. These success indicators are not what people expect. They don’t point to the majority, large numbers, or a set of fixed, testable criteria. They don’t refer to the people who have the authority, influence, or popularity. There is also no best. Success is relative. It isn’t on a single scale. Everyone is different, every system is different, every point in time is different, every measure is different and changing. Part of inclusive design and We Count is to re-think how we measure success.

All my best,

Jutta

**Please note this work is licensed under an Attribution-NonCommercial 4.0 International License**

Through a project called We Count, my team and I pulled together an amazing community focused on ensuring that people, who are different from what is deemed average, are not excluded, ignored, misunderstood, mistaken as threats; and their data is not abused, or misused by emerging systems. We are specifically concerned with decision systems, artificial intelligence, and data systems that rely on large amounts of data to arrive at predictions, a simplified truth for the majority, or an efficient judgement that ignores exceptions. As part of this project, we published a newsletter, and I wrote a “Director’s Message” for each newsletter. I am sharing these messages here as a series of blogs.

Dealing with a world that is not designed for you can be a frustrating, even soul-crushing, experience. Simple things like eating, joining a conversation, finding a home, clothing your children, finding a bathroom, reading an announcement, getting your message across, finding information, or getting anywhere, become difficult if not impossible activities. Our economies and distribution of wealth make these tasks even harder. If standard products and services are not designed with your needs in mind, you need to buy specialized products and services. You can’t benefit from economies of scale, and everything comes at a premium cost. If you think housing is expensive and hard to find in your city and prices are constantly rising due to demand, for example, try to find even one accessible home or apartment and think about all the people that need these rare finds. Setup, training, maintenance, updating, repair, help and all other services are also scarce and more expensive. Think of this as a maddeningly unfair disability tax.

If you have the stamina not to be crushed by this unfairness, it gets worse. Unemployment rates of people with disabilities, willing and able to work, is higher than any other group. Employers willing to hire and support you are also rare and hard to find. Government financial aid keeps you well under the poverty line despite your higher costs (to add to the insult, the media creates memes of people abusing the welfare system). Even the federal government Market Basket Analysis on poverty reduction (that calculates the different costs of living across the country) only makes a passing reference to the mention of disability in the consultation and doesn’t give it as much priority as communication technology costs.

All of this is made worse by the adoption of artificial intelligence decision tools in education, employment, research & data analysis, policy guidance, marketing, social media, news programming, banking & financial institutions, medicine, and all other decisions affecting our daily life. In these decision tools, difference from the statistical average or majority means you are not recognized, understood, or considered in the decisions. The historic, systemic unfairness is compounded.

Through projects like We Count, we are trying to address this bias, by increasing awareness; teaching data systems to recognize, understand and support the way we are all different; and teaching them not to multiply the maddeningly unfair treatment if you are an outlier or a minority. We welcome your help and participation in these efforts.

All my best,

Jutta

**Please note this work is licensed under an Attribution-NonCommercial 4.0 International License**

Through a project called We Count, my team and I pulled together an amazing community focused on ensuring that people, who are different from what is deemed average, are not excluded, ignored, misunderstood, mistaken as threats; and their data is not abused, or misused by emerging systems. We are specifically concerned with decision systems, artificial intelligence, and data systems that rely on large amounts of data to arrive at predictions, a simplified truth for the majority, or an efficient judgement that ignores exceptions. As part of this project, we published a newsletter, and I wrote a “Director’s Message” for each newsletter. I am sharing these messages here as a series of blogs.

February 2022

Your Attention Please!

Our attention has become one of the most fought over commodities. As the things that demand our attention proliferate, both our self-determination and the quality of what we attend to tends to deteriorate. The art of manipulation and stealing attention have become normalized standard practices, even by trusted authorities. Surveillance tools that track what we pay attention to have been embedded in the systems we rely on daily, such as email, social media, and our Web browsers, but also through tracking systems in public spaces.

In this fierce competition for attention many of the things we value are at a disadvantage. Honesty, transparency, thoughtful debate, even optimistic good news loses out to sensationalism, insults, alarmism, and exaggerated promises. Group think, flashpoints and popularity contests become more powerful than respect and dignity.

If you experience a disability, you have likely felt the brunt of the risks and abuses of this competition. Topics relevant to the life of a disabled person generally disappear because they are not popular, and attention breeds attention. If relevant topics rise in popularity, it is often because the narrative has been dramatized to appeal to disempowering attitudes toward disability. Heroism, sensationalized victimization, and demonization are powerful attention hooks. Where disability identity has gained respect, it may be used by others to polish the attention they receive. Rarely are you able to own and tell your own authentic story, and if you can, it doesn’t get the attention it deserves.

It is not surprising that data tools are employed in this competition for attention. If you are a small minority or an outlier, these tools put you at a disadvantage. AI is used to determine what is displayed, where, and when, based on what can garner the greatest attention and the greatest profit, without regard to other impacts. Often the topics you care about can’t compete. The damage to you is ignored or marked as trivial. Personalization systems that filter and curate the information you receive come with two potential harms. The assumptions used by the AI to choose the information you are exposed to work well if you are part of the majority with interests that are more common, but don’t work well if the AI has not encountered or cataloged the topics you care about. Secondly these systems tend to restrict exposure to new, surprising, and possibly dissonant views, meaning that our society closes in and becomes less tolerant of difference.

In project We Count and at the Inclusive Design Research Centre we have been exploring ways to counter bias against disability and difference for many years. We are looking at AI systems and algorithms that encourage exploration and exposure to difference, using authentic data relevant to our diverse and collective perspectives, and under our control. We are co-creating tools that can be used to tell our own authentic stories. We would love your help in co-designing these new systems.

All my best,

Jutta

**Please note this work is licensed under an Attribution-NonCommercial 4.0 International License**

Through a project called We Count, my team and I pulled together an amazing community focused on ensuring that people, who are different from what is deemed average, are not excluded, ignored, misunderstood, mistaken as threats; and their data is not abused, or misused by emerging systems. We are specifically concerned with decision systems, artificial intelligence, and data systems that rely on large amounts of data to arrive at predictions, a simplified truth for the majority, or an efficient judgement that ignores exceptions. As part of this project, we published a newsletter, and I wrote a “Director’s Message” for each newsletter. I am sharing these messages here as a series of blogs.

December, 2021

Dear Community,

We need your help, especially from those of you with experience of barriers to employment because of a disability. We are embarking on a project called ODD, standing for Optimizing Diversity with Disability. (We intentionally chose the title ODD to reclaim a term that has been used pejoratively in the past, but that actually signals a valuable trait.) The goal of the project is to document bias against disability in artificial intelligence (AI) hiring tools and find a way to address this. More and more companies are relying on AI tools to choose applicants to interview or hire.

AI hiring tools have filters that are designed to replicate past successes using past employment data. The filters will find applicants that best match current and past successful employees. These tools also offer employers options to find applicants that attended the employer’s favourite university, that fit the current culture of the organization, that fit a persona of an ideal employee, or that fit into a profile of the ideal team. All these filters are biased against diversification of the team and toward a monoculture. This is bad for the organization but especially bad for applicants with disabilities.

Our plan is to co-create what is called a test “synthetic data set” with your help. This means a set of data that is based on profiles or resumes of co-creators who live with diverse disabilities; without information that can identify the person. To get around privacy concerns we will mix up some of the identifying traits, so that re-identification will not work. But we will keep the important data that might flag a resume or profile as not a good fit with the AI hiring tool. (For those of you interested in privacy protections, this is better than Differential Privacy as it doesn’t remove the data needed to serve someone with unique qualities or needs.)

You do not need to have prior knowledge of artificial intelligence, privacy protections or hiring tools to participate. All we need is people who apply to jobs but experience barriers in getting hired.

We recognize that getting hired is only the first step. The critical part of employment is a place of employment that supports full participation. Being hired without this would set you up for failure. We will also co-design and co-curate a set of resources needed to create an inclusive workplace.

We hope you will join us in fighting the abysmal unemployment and underemployment rate of disabled people who are ready to work.

All my best,

Jutta

PS: We continue to build out our synthetic data set.

**Please note this work is licensed under an Attribution-NonCommercial 4.0 International License**

Through a project called We Count, my team and I pulled together an amazing community focused on ensuring that people, who are different from what is deemed average, are not excluded, ignored, misunderstood, mistaken as threats; and their data is not abused, or misused by emerging systems. We are specifically concerned with decision systems, artificial intelligence, and data systems that rely on large amounts of data to arrive at predictions, a simplified truth for the majority, or an efficient judgement that ignores exceptions. As part of this project, we published a newsletter, and I wrote a “Director’s Message” for each newsletter. I am sharing these messages here as a series of blogs.

Friends and colleagues,

With the proliferation of misinformation, disinformation, and manipulative half-truths, we have felt the need to double down on, and defend truth. Recognizing lies is critical to our survival. The target of this misinformation is often the members of our community. However, I worry that in the defense of truth we as a society have dangerously reduced truth.

We as a society have reduced truth to the truth for the majority and the statistical average. No truth is deemed accurate or provable unless it can be statistically verified. This denies, trivializes, or ignores the truth for the minority or outlier. At worst it demonizes these truths as false, unprovable, or merely anecdotal.

We as a society have also reduced truth to only measurable or quantifiable facts. This creates an impoverished truth that denies or devalues all the parts of our lives that cannot be counted or measured. Related to this is the explicit or implicit notion that the truths that are most relevant are the truths that can be monetized. What happens to the things in our lives that are priceless?

We as a society have also reduced truth to the truths of the apparent or obvious facts. The Canadian author Farley Mowat famously insisted that he “never let the facts stand in the way of the truth.” Some people have interpreted the assertion as a defense of “making up things” and “stretching the truth.” Mowat saw it as a way of getting at the essential truth. He felt that facts can be misleading; to arrive at truth you need to look beyond the facts and intimately know something or someone from many perspectives. People who are judged and condemned by a single snapshot on social media likely agree with this sentiment.

In requiring data to support truths we have also constrained our view of truth to the truths of the past. We have made assertions of truths more deterministic of the future. We have accepted the assertion that what is true about you in the past is what will be true of you in the future. Anything else is seen as inauthentic, posing and illusional. Recommender engines assume that what you are interested in or like in the past, determines what you are interested in and like in the future. In this we have reduced our exploration of potential. Adrianne Clarkson in her Massey Lecture on belonging and citizenship points to the “as if” philosophy of Hans Vaihinger. By acting as if a desired truth is so, we help to make it a reality. Good teachers know this trick. Treat students as responsible and smart, and they tend to meet your expectations. Treat them as hopeless and misbehaving, and they tend to meet your expectations as well. This phenomenon is a “proven verifiable fact” called the “self-fulfilling prophecy.”

The creation of a reduced truth, and the defense of this reduced truth, has a ripple effect for any justice seeking groups and individuals. It supports austerity movements. Only factually supportable requirements are deemed defensible. This also reduces the approaches we consider for addressing some of the most critical problems we are facing. For example, inexplicably, a picture of a beautiful green landscape in an intensive care unit (ICU) will measurably reduce the need for pain medication and the length of stay in an ICU. The women’s suffrage movement asserted that we all have a right to both “bread and roses,” to feed our souls and minds, not just our bodies. One of the unmeasurable criteria of truly accessible design is that it be beautiful design.

In the We Count project we are exploring approaches to defend truth without reducing and impoverishing it. We welcome your help and suggestions.

All my best,

Jutta

**Please note this work is licensed under an Attribution-NonCommercial 4.0 International License**

Through a project called We Count, my team and I pulled together an amazing community focused on ensuring that people, who are different from what is deemed average, are not excluded, ignored, misunderstood, mistaken as threats; and their data is not abused, or misused by emerging systems. We are specifically concerned with decision systems, artificial intelligence, and data systems that rely on large amounts of data to arrive at predictions, a simplified truth for the majority, or an efficient judgement that ignores exceptions. As part of this project, we published a newsletter, and I wrote a “Director’s Message” for each newsletter. I am sharing these messages here as a series of blogs.

Friends and colleagues,

What is intelligence? The Merriam-Webster Dictionary definition is: “The ability to learn or understand or to deal with new or trying situations. Also: the skilled use of reason.” What we consider to be human intelligence appears to have created a swarm of trying situations that we do not know how to deal with. Is it time to reconsider what we mean by intelligence? We have had one expansion of intelligence when we admitted that there might be another dimension to intelligence, namely emotional intelligence. The Psychology Today definition is: “Emotional intelligence refers to the ability to identify and manage one’s own emotions, as well as the emotions of others.”

Neither of these dimensions of intelligence appear to have considered the broader or long-term consequences of our decisions. Our reasoning has been short-ranged and over-simplistic. Our strategizing has been enamored of reducing considerations to numbers and complex factors into cleanly boxed categories, ignoring the entangled connections and relationships.

Our intelligence, even when expanded to include emotional intelligence, appears to be bereft of a global view and an ethical compass. We seem not to be cognizant of the simple fact that we are all connected and living on a single planet. That what goes around, comes around. We can’t recognize the importance of maintaining our diversity or learning to navigate complexity. Because we have created artificial intelligence in our image, it has only amplified, accelerated, and automated our deficient form of intelligence. It has become a power tool that speeds us to further trying situations. At project We Count we hope to do a small part to create a more complete, long-term, broader definition of human and machine intelligence.

All my best,

Jutta

**Please note this work is licensed under an Attribution-NonCommercial 4.0 International License**

Through a project called We Count, my team and I pulled together an amazing community focused on ensuring that people, who are different from what is deemed average, are not excluded, ignored, misunderstood, mistaken as threats; and their data is not abused, or misused by emerging systems. We are specifically concerned with decision systems, artificial intelligence, and data systems that rely on large amounts of data to arrive at predictions, a simplified truth for the majority, or an efficient judgement that ignores exceptions. As part of this project, we published a newsletter, and I wrote a “Director’s Message” for each newsletter. I am sharing these messages here as a series of blogs.

September 2021

Dear friends and colleagues,

This week I had the opportunity to speak at an international forum attempting to craft policies for a global accord regarding fundamental rights in AI and digital societies. I wanted to share my message here. I would be pleased to explain any of the points I make. We explore these same points in the many activities of the We Count project.

“I’m worried about the collateral damage of AI, even when it is deemed to be ethical and fair; even when we have full participation and access; when it is used by good actors, not only exploited by bad actors; and well before it can pass the Turing test.* My work is with people who tend to be at the margins of any population data set, people who are outliers or tiny minorities. Among these are people with disabilities who are found at the edges of all other justice seeking minorities, do not form an identifiable data cluster because of their diversity, but collectively make up the world’s largest minority.

The most pervasive AI, at the moment, is not the helpful or harmful robot that has captured our imaginations, but a mundane but ubiquitous power tool that currently either makes or guides choices in most of the critical areas of our lives.. who is hired, what medical treatment we receive, whether we get a loan or credit, whether we get into college, whether we are a security threat, whether our concerns receive attention in media or in politics… and many other life-changing decisions. I’m worried about the AI tools that are deployed to make human choices more efficiently, following the formula it is presented more accurately, at a faster rate and at a larger scale…the AI that is used to optimize past successes. These AI tools will worry me even when there is full proportional representation and we have removed human bias from the algorithms and the datasets.

AI, in optimizing past patterns of success more efficiently, also amplifies, accelerates and automates past discrimination. Because of the efficiency, speed and scale of AI, and because of the vicious feedback loops it sets in motion, this discrimination increases exponentially with each cycle of machine learning, speeding us to greater disparity and pushing us to greater homogenization.

The discrimination I’m speaking of is more than the obvious human racism, sexism, or classism. It is discrimination we find in our revered scientific evidence and in democratic elections, even with full proportional representation. It is a by-product of decisions based on statistical averages, majority rules, popularity metrics and economies of scale. It happens in even the most progressive societies. In fact, the more well-off a society becomes the more likely the trivial needs of the many will outweigh the critical needs of the few.

Before these seemingly mundane decisions were mechanized using AI, we could make exceptions, appeal to qualitative human judgements (that marred the quantitative perfection). With current AI this possibility is removed, and AI exponentially amplifies the resulting disparity.

People who are outliers and small minorities have the dilemma of choosing not to participate in a system that will not benefit them or participating to make sure they are represented and not completely excluded and ignored.

Even if governments are willing to accept the collateral damage of technological progress; diversity and difference are other casualties. Without them we have monocultures. It is time to create human and mechanized decision systems that support diversity, can navigate complexity and can respond to change.”

all my best,

Jutta

*Note: The Turing test is intended to determine whether AI can adequately mimic human intelligence to fool an observer.

**Please note this work is licensed under an Attribution-NonCommercial 4.0 International License**

Through a project called We Count, my team and I pulled together an amazing community focused on ensuring that people, who are different from what is deemed average, are not excluded, ignored, misunderstood, mistaken as threats; and their data is not abused, or misused by emerging systems. We are specifically concerned with decision systems, artificial intelligence, and data systems that rely on large amounts of data to arrive at predictions, a simplified truth for the majority, or an efficient judgement that ignores exceptions. As part of this project, we published a newsletter, and I wrote a “Director’s Message” for each newsletter. I am sharing these messages here as a series of blogs.August 2021

Dear Friends and Colleagues,

Humans have always had a hard time with difference, the stranger, or the unknown. Our society prides itself and trains our kids to find things that match a familiar thing or person, and to sort and categorize things or people into familiar categories. Humans have difficulty with the unexpected or things that don’t fit our categories. In setting targets for designing artificial intelligence and measuring ultimate success, developers have set the goal of matching and surpassing human intelligence. The ultimate test is the Turing test which is passed if a person can’t tell the difference between human performance and AI performance.

It is therefore no wonder that AI matches our strengths as humans and makes these skills more efficient, faster and more accurate, but also matches our weaknesses and intensifies the harm. This harm is showing up in the many ways we have asked AI to make choices for us — to sort the desirable from the undesirable, and the things that are worthy of attention from the things we can ignore.

One of the harms of these human and AI weaknesses, is that they encourage monocultures. Monocultures will ultimately result in the demise of the business, society, culture or race. A monoculture can be felled with a single threat, has limited perspectives to detect real risks, and has a limited set of choices for changing direction.

AI is being used to sort choices to:

• find potential employees worthy of an interview or hiring, from resumes and candidate profiles
• pick applicants for competitive academic spots
• determine what people should pay attention to in the news and in their reading material
• choose immigrants to allow into the country, and
• decide who should receive a mortgage or credit.

In every case, AI uses data about the familiar and the profile of the current favourite, thereby fine tuning the path to a monoculture.

AI treats the novel, unknown or anomalous as a threat, flagging anyone that differs for:

• extra screening at a border crossing
• audits for tax evasion, or
• potential cheating in an online exam.

At the We Count project we are working to create systems that recognize the value of the novel, the different, the unknown and unexpected. Difference expands our strengths, builds out the perspectives to recognize potential risks, and provides a richer set of choices when the world acts in unexpected ways. From resume filters that highlight new perspectives, to Word clouds that emphasize the unique, to systems that indicate when AI should not be used to make decisions about someone for whom there is little data; we are pushing against monocultures. We invite you to help co-design systems that value difference.

all my best,

Jutta

**Please note this work is licensed under an Attribution-NonCommercial 4.0 International License**