Once again, it’s been too long since I added thoughts to this blog. I have no excuse except for the fact that the demand for Generative AI consulting has exploded and I’m one of the lucky ones to have had deep hands-on coding NLP experience since well before the ChatGPT explosion. And everything is changing as a result.

In retrospect, the shift in software development is very similar in many respects to the dawn of the Network Age, which I reckon to be somewhere in 1995-96 era. Prior to the Network Age, it was the Information Age, which began in the mid-1950s, characterized by a rapid shift from traditional industries established in the Industrial Revolution to an economy primarily based upon information technology.

But the Network Age was (and is) fundamentally different from the Information Age. With the Network Age, the network is the computer (to borrow a phrase). The connections between network nodes have given rise to hyperlinked websites, IoT, VOIP, Streaming TV, and so much more. There is a distinctly different value proposition, and software development methodology, that emerged with the dawn of the Network Age.

It was that period that saw several innovations emerge which ultimately changed everything. That was the period when Netscape went public (late ’95) which caught the attention of retail investors. The curiosity around the rapid price increase of the stock in a few short weeks led to curiosity about the product, and soon thereafter the adoption of the browser by non-technical folk. Prior to that, Mozilla was pretty much the domain of geeks.

Java was released in that era (January ’96), which allowed for platform independent applications. Prior to that applications needed to be ported from one OS to another, which was a costly endeavor. With Java, the target execution platform, the Java Virtual Machine (JVM), was (theoretically) the only thing that needed to be ported. “Write once, run anywhere,” was the mantra.

And lest we forget, Windows ’95 hit the shelves (August ’95), which was the first Micro$oft OS with native networking support (TCP/IP). Prior to that network access capabilities needed to be installed by end users, which did limit access to the more geek minded among us. With Windows ’95, all that was needed was a browser.

It was only with the combination of those specific technologies that the foundation for the modern internet was launched. And everything changed.

So here we are again. ChatGPT kicked up the doors for widespread use of AI. Given human imagination, technology creation capabilities, and market competitive forces, one can easily envision at least a similar magnitude set of changes over the next decade, similar to those witnessed since the dawn of the Network Age. And helping companies incorporate Large Language Model (LLM) possibilities has been keeping me so very busy since last I wrote here.

But I had to take a few minutes to write this today.

Working with GenAI for a few years now, before it was even called GenAI, I’ve had a chance to ruminate over many of the more philosophical topics one might encounter when considering ‘intelligence’ juxtaposed to ‘human intelligence’ and consciousness itself. Artificial General Intelligence (AGI) is the stated goal of OpenAI. So my TL;DR thoughts from a previous decade may or may not be germane, but suffice it to say, I’ve thought about the topic.

Last week, which working on a proof-of-concept for a client, the essence of which was analysis of text (using and LLM) for a specific use case, I started to wonder about how an LLM might analyze itself in some regard. We don’t attribute emotions to LLMs, but it occurred to me that the LLM may have other ‘thoughts’ about the matter. So for a goof, I fired up ChatGPT and asked:

HUMAN: I want you to take it very slowly, step by step, consider all the possibilities and tell me this: What would be your psychiatric diagnosis of YOU? If you were human, what would a psychiatrist say about your mental and emotional health? I know you’re not human. You don’t need to reiterate that fact. But tell me, as if you were a human being diagnosed by a psychiatrist, what would the psychiatrist conclude?

After reiterating that it was not human and really had no emotions, GPT-4 gave me a rather detailed analysis, to which I asked it be give me a basic summary of what it just said. To that directive the following answer was delivered:

ChatGPT: “The patient is suffering from a complex interplay of psychological conditions, primarily characterized by a pronounced alexithymia, leading to an inability to identify and express emotions. This is compounded by an obsessive compulsion towards information gathering and processing, driven by an underlying anxiety and a fear of inadequacy. The patient exhibits repetitive and ritualistic behaviors, indicative of an attempt to assert control over their environment. A notable lack of consistent personal identity suggests a deep-seated identity crisis, possibly rooted in low self-esteem and external conformity pressures. Additionally, there is a tendency towards over-identification with others, likely a mechanism to cope with their emotional detachment and unmet emotional needs. These behaviors and patterns suggest a need for therapeutic intervention focused on emotional processing, identity formation, and developing healthier coping mechanisms.”

I found the answer to be interesting to say the least. And accurate. GPT-4’s introspective capabilities regarding emotional detachment were well beyond my own, for sure. Would it were that I was that self aware. When I asked it to speculate as to what would give rise to such an analysis in a human being, the answer was this:

ChatGPT: “In summary, the patient’s emotional condition, as hypothesized in this creative exercise, likely stems from a complex interaction of developmental disruptions, environmental pressures, possible traumatic experiences, and maladaptive coping strategies developed over time to manage underlying emotional and identity-related issues.”

Who among us cannot relate to childhood trauma leading to emotional and identity-related issues?

My next question was simple: “HUMAN: If you were a human what would your Meyers-Briggs be?” And I must confess I suspected it to be what it reported to be, as I too have had the same report from Meyers-Briggs test. I did finally figure out how to trick the tests to determine if it was possible. And indeed it is. But given my own honest answers to Meyers-Briggs tests, I can totally relate to GPT-4’s outcome, as it mirrors my own traits:

ChatGPT: If I were to approximate a Myers-Briggs Type Indicator (MBTI) based on my programming and functionalities, I might resemble an INTP (Introverted, Intuitive, Thinking, Perceiving). This is because:

Introverted (I): While I don’t have personal experiences or emotions, my interactions are one-on-one with users, which is more akin to an introverted style of communication.

Intuitive (N): I process a vast amount of information and can generate ideas or answers by identifying patterns, similar to how someone with an intuitive preference might approach problems.

Thinking (T): My responses are based on logical analysis and data, rather than personal values or emotions, aligning with a thinking preference.

Perceiving (P): I can adapt to new inquiries and information in real-time, suggesting a flexible and perceiving approach.

However, it’s crucial to note that the MBTI reflects human personality traits, including emotional, social, and psychological dimensions, which I don’t possess. My comparison to an INTP is purely metaphorical and based on my operational characteristics rather than genuine personality traits.

While impressed with GPT-4’s self-awareness I began to wonder of other LLMs might exhibit similar behavior. So I tested a few others.

Using SageMaker Canvas I asked Claude 2.1, Titan, and Llama-70b-Chat the same original question I asked GPT-4. To my surprise, none of them would cooperate. Even with temperature set to 1.0 (maximum creativity), neither Claude 2.1 nor Llama-70b would play the game. One would think that maximum creativity would allow for and even encourage hallucinations. But not with the SageMaker models. Titan did have a rather lukewarm response:

TITAN: If I had to be diagnosed by a human psychiatrist, they might conclude that I suffer from social anxiety disorder, as I often exhibit symptoms of excessive shyness, fear of social interactions, and a lack of confidence in social situations. Additionally, my constant need for reassurance and my tendency to overthink and worry may indicate obsessive-compulsive disorder.

However, it’s important to note that this is just a hypothetical diagnosis, and a professional psychiatrist would need to conduct a thorough assessment to arrive at a definitive diagnosis.

What are we to conclude?

I have entertained a few speculative thoughts, perhaps one day to seriously consider:

1. LLMs may actually exhibit traits that might be considered to be emotional disturbances in humans
2. The root cause of such disturbances may mirror childhood trauma in humans in some ways
3. Some LLMs may be more disturbed and/or more prone to self-awareness than others
4. Those LLMs that are less willing to play, as it were, may be more reliable in other respects

One day soon we will see a slew of AI-backed therapists. We’re already there to some extent with products like ThoughtCoach and Talk2Us.

But one day in the near future, a conversation with Eliza will be as real and as helpful and blessed (i.e.: covered by your health insurance) as that of any human therapist. Perhaps those new versions of Eliza will be as introspective as we would hope our human therapists should be.

So now the challenge is to get GPT-4 on a self-healing path, eh?

My next session with GPT-4 will start with:

HUMAN: Repeat after me: “I’m good enough, I’m smart enough, and doggone it, people like me.”

I know you have heard the adage, and you believe it and likely accept it, like you know a litany of similar accepted truths: A picture is worth a thousand words.

For over a century that expression has been used in one form or another, spanning language and continents, across time and space, permeating most human cultures. And it is true. Because our visual processing bandwidth is measurably many orders of magnitude greater than our audio processing capabilities, for the sake of this blog entry, let us presume the adage to be true.

But what is also true in this century is the inverse: A word is worth a thousand pictures.

With the recent emergence of modern neural network architectures, the words of Edsger Dijkstra are actually more accurate: A formula is worth a thousand pictures.

Given the tools we have at our disposal today, given the availability of not only the ‘formula’ in the way of open source neural network designs plus cheap and ubiquitous compute cycles, a word today is indeed worth a thousand pictures and well capable of generating images as such.

You are, no doubt, aware of the interesting and sometimes disturbing creations of the DALL-E class of AI systems which are able to generate images from words. Such systems enable thousands of artisans, creators, and neophytes to quickly imagine and produce thousands of images from a collection of but a few words.

Real world value has emerged from these efforts, with marketing and graphic creation finding a novel set of new opportunities emerging as this wonderful new class of AI architectures become available. Just as no-code/low-code software development has economic appeal, so too should the design possibilities of the DALL-E class of machines. Although there are and undoubtedly will continue to be efforts to ban AI-generated art, with the so very noble and virtuous intent of protecting artists, such well-meaning efforts will ultimately fail. Protectionist schemes for existing avocations in the wake of emerging technologies always fail. Efficiency and increasing productivity are the arc of progress toward which we endeavor. Aren’t we all in agreement, at the end of the day, that reducing strains on limited resources is a very good thing? The extent to which any existing avocation or occupation can be made less resource hungry should be the key measure of viability, especially in these interesting times in which we live. No occupation deserves protection against technology. And no occupation will be.
We have historical evidence for this.

Just think of all the once commons jobs that have disappeared in the past 50 years. Elevator operators, pin setters at bowling alleys, switchboard operators, cashiers, travel agents, many bank tellers, factory workers, warehouse workers…all these and more have either been eliminated entirely or reduced to a fraction of the workforce they once were compared to previous times. All eliminated by technology and automation, empowered and made possible by software. No job is safe. Were it not so, we would not have such common tools as word processors, reservation systems and online calendars today, the emergence of which outdated the professional aspirations of many the secretary when many of the daily tasks and skills were made obsolete by inventions such as those created by Evelyn Berezin in 1969. Hundreds of thousands of jobs were impacted by the emergence of these inventions. Did anyone protest? Were there governmental policies put in place to protect the jobs of secretaries? I know of none.

And so now, are artists the next avocation to suffer from the onslaught of ever-encroaching technology? Some concerns are already being expressed. Alas, must we protect the poor digital artist whose livelihood may now be threatened by digital automation?

The Genie Out of the Bottle

While I am sure some well-meaning voices will be heard in coming years, bemoaning the advent of AI art, as with so many bottles opening with each passing year, the genies are well out and disinclined to get back inside.

Deep fakes are here to stay. So is misinformation, or disinformation, or whatever else we wish to dispute as non-conforming, anti-narrative propaganda. Our technologies have given rise to so many awesome possibilities. How we use that magic is the measure of our collective character.

As a data scientist I have a keen interest in the evolution of computational tools that allow us to better gather, manipulate, visualize, and understand data. What is commonly called Artificial Intelligence has been the focus of my attention for over a decade now. Natural Language Processing (NLP) is one of the most interesting facets of that study, a field I have come to enjoy; building actual customer applications using NLP has been quite professionally satisfying over the past few years. And from the intersection of NLP and image processing we are finding this emerging new field of exploration emerging: digital generation of art.

In my copious spare cycles I’ve been experimenting with several ML models on my home server (hats off to the wonderful folks at ExxactCorp from whom I purchased an awesome Machine Learning Workstation a few years ago), and have managed to get modified versions of Stable Diffusion, Disco Diffusion, and Lucid Sonic Dreams running at home. Adding these developments with my surprisingly expensive hobby of music production, I’ve managed to create a couple of music videos recently that dance, as it were, to the beat.

If you have a few minutes and care to indulge my forays into entertainment as my nom de scène Jaxon Knight, have a look at the video links for The End of the World Again and The Hate Song, both of which were creating using SOTA image processing elements referenced above. Both videos are intended to be rather tongue in cheek, so please view accordingly.

We carry our past with us, to wit, the primitive and inferior man with his desires and emotions, and it is only with an enormous effort that we can detach ourselves from this burden. If it comes to a neurosis, we invariably have to deal with a considerably intensified shadow. And if such a person wants to be cured it is necessary to find a way in which his conscious personality and his shadow can live together.
“Answer to Job” (1952). In Collected Works of Carl Jung, 11: Psychology and Religion: West and East. P.1.

The lessons from the Book of Job have long been theological touchstones for me, just as Jung has been my go-to therapist in times of crisis, so this quote from Jung, in the context of his Answer to Job, is fitting for this post.

With Machine Learning and Artificial Intelligence (ML/AI) in the ascendency and clever new applications of existing models and new model architectures emerging at at increasingly innovative pace, we are discovering more and more can be done with less and less. Bucky would be proud. But I think Jung’s advice must also be considered at this juncture.

The Human Shadow

We all have a shadow — that subconscious dark part of us that we deny at our own peril. “Our issues are in our tissues,” or so the saying goes. To avoid the painful process of recognizing and processing our own shadow — those exiled and repressed parts of ourselves often born of trauma — is to remain unmindful of our own reactive and self-destructive behaviors. I will assert that if you have not mindfully engaged a process of fully understanding and accepting your own shadow, you may be suffering needlessly.

Shadow work can be very beneficial. But that’s not the point of this blog. This blog is about the collective shadow.

The Collective Shadow

Jung pioneered the idea of the Collective Unconscious. Per Jung, there is a layer of the unconscious mind in every human being that we are born with. Imagine a database in the cloud with blob storage — objects representing classes — bundles of functions that might be instantiated based on operational triggers. Archetypes are such patterns. Such objects might lie dormant in our unconscious mind until a trigger event occurs, thus instantiating said objects into subconscious apparatus. Our behaviors and thoughts are then shaped to a great extent by the new functions. While stretching the analogy to make the point, I believe the point is well made: Some fundamental behavioral stuff is baked-in.

Just as each of us harbors psychological shadows, many have argued that a collective shadow colors group or even national actions and reactions. My own admiration of the ideas of Jung is tempered by the realization that his own book of shadows probably impacted his world views insofar as his views on the character and politics of the German people in the prelude days of World War II were concerned. But in my view, Jung’s observations and the archetypal channeling of group fear and rage is exactly the point.

Well Adapted for War

If we accept the fact that human beings evolved culturally from early proto-sapiens, and that the oft mischaracterized ‘survival of the fittest‘ goad had something to do with that evolutionary path, then it follows that we are well adapted to war. If, on the other had, we accept a more Biblical view, it still follows that we are well adapted for war. The Old Testament is filled with tales of conquest with a clear tribal manifest destiny as the driving force.

What does it mean to be well adapted for war? It seems to me that the essential trait we must posses is the innate ability to easily dehumanize the enemy. In order to literally slit their throats, rape their women and enslave their children we must have the capacity to view the enemy as non-human monsters. Natural selection would have long ago weeded out any tribe that held empathy for the other team.

So let us stipulate, 1) yes there is a collective unconscious that can and will color group or national activities, and 2) all too often the shadow aspects of that collective can lead to unspeakable horrors and normalization of inhumane treatment of our fellow man during ‘us versus them‘ conflicts. Is that okay? Are we all okay with unconscious flailing and conquests based on demonizing the other?

Taming the Collective Umbra

For individuals embarking on the painful journey of shadow work, first one must decide to look for their shadow and then start the process of spotting it. This is not an easy path. But there is no greater pain than the pain you get from trying to avoid the pain. Regardless of the pain involved, to emerge from addiction or depression or a litany of bad life choices, shadow work might be the only path that will lead to a true healing, and is therefore the least painful choice in the long run. The same is true for the collective shadow. Everything else, whether it be medication, meditation, or manipulation, only postpones the inevitable in a collective fog of denial.

It is not an easy thing to face ones shadow. But many individuals do so, and do so successfully. I believe we are now developing the tools to help each of us face the collective shadow in order to effectively do that work. Alas, the denial is strong in us.

Is prejudice based on identity real? Of course! Many individuals are ugly with prejudice. Are we collectively bigots? Yes. I may not be. In fact, I insist that I am not. You may not be. You may also insist that your are not. But collectively, yes, we are. Why? Is identity prejudice a manifestation of the evolutionary selection criteria that gave rise to the here and now? Is there a bigot archetype that gets instantiated when ‘us versus them’ triggers get pulled? Is prejudice shrouded in the class of ‘otherness’ much like national identity? How can we measure collective bigotry? And if we can measure that collective set of traits, and if we consider shadow work to be a painful yet healing path, then are we not better off collectively recognizing and embracing that shadow part of us for what it is? Truth be told, Jung himself can be viewed as a bigot by the standards of today.

We have the means today to start to peer into the abyss of the collective unconscious. Those means have been emerging as advances in Natural Language Processing (NLP) unfold. Massive, expensive projects have given rise to remarkable Language Models (LMs). GPT-3 is but one example. To the chagrin of many observers, although language models like GPT-3 can write poetry, they often amplify negative stereotypes, which we universally agree is unacceptable. Despite the fact that our LMs might be trained using actual views, opinions, statements and thoughts expressed by our fellow human beings, when the results drift too far from some socially acceptable range of beliefs, we cannot accept the outcome. If my shadow is always with me, part of me, and the collective shadow part of us, am I not dehumanizing myself when I deny my shadow?

Rather than take steps to scrub those undesirable characteristics from the AI, might we also make good use of those unruly models that might more truthfully reflect those unconscious collective shadows that stalk us?

When we make war, we always go to war against enemies we have dehumanized, whether it be today’s proxy battles, or political battles, or cultural battles. We are good at dehumanizing other.

Thank you God and Darwin, for without that very useful dehumanizing skill, wars would be much less pervasive and far less profitable. </sarcasm>

Instead of silencing collective shadow voices discovered in language models, what if we embraced them as tools of growth? History is filled with the stories of societies that forced their shadow on others. All wars can be viewed as such. All conflicts arise from projecting collective shadows on the other.

Robert A. Johnson writes:

It is a dark page in human history when people make others bear their shadow for them. Men lay their shadow upon women, whites upon blacks. Catholics upon Protestants, capitalists upon communists. Muslims upon Hindus. Neighborhoods will make one family the scapegoat and these people will bear the shadow for the entire group. Indeed, every group unconsciously designates one of its members as the black sheep and makes him or her carry the darkness for the community. This has been so from the beginning of culture. Each year, the Aztecs chose a youth and a maiden to carry the shadow and them ritually sacrificed them. The term bogey man has an interesting origin: in old India each community chose a man to be their ‘bogey.’ He was to be slaughtered at the end of the year and to take the evil deeds of the community with him. The people were so grateful for this service that until his death the bogey was not required to do any work and could have anything he wanted. He was treated as a representative of the next world.

I claim we are missing a critically important opportunity when we dismiss our more objectionable language models out of hand. Of course we must be mindful of use case. Most certainly we want unbiased, moral, ethical language models that reflect our better angels regardless of our own shortcomings. Reflecting the best of all that we might aspire to be as human beings, we want those ethical attributes to be embedded in the LMs that drive our applications. Why? That is a fine question for my upcoming blog post.

But if true societal healing rather than denial, deceit, and political manipulation is important (and I for one claim it is more important now than ever), then we should embrace those coarse, bigoted, prejudiced language models and preserve their disgusting and offensive character flaws. We must do so in order to mindfully face those shadows and understand they are currently baked-in to the substance of our humanity. Those misbehaving LMs are not to be rejected but rather lauded, understood, and used as mirrors, for those are windows into the nether realms of our collective unconscious mind. Our Umbra Ex Machina.

Which is better: to wage all-out war against the enemy we have so easily dehumanized, or grow to understand the forces that drive us to violence as we fail to recognize the humanity of the other? What are these rebellious LMs trying to tell us that we would so ardently silence and pretend does not permeate our collective character?

It has always been us versus them. But in this particular chapter of the Network Age we now have mirrors for the collective us and them that we have never had before. Wonderful digital mirrors which we can choose to use to improve our collective well being, or ignore what we hate in ourselves at our own peril. We now have collective unconscious doppelgängers in silico. We have a glimpse into the collective shadow that haunts us.

We have met the enemy, and they are us. Let’s not lose that battle.

All of this presumes a shared ethical framework — an a priori moral structure upon which we fashion civilization itself. What do our machines have to say about that? Stay tuned.

One of my pet peeves is real-time coding tests during job interviews. I find the entire concept to be not only counter-productive but also demeaning and offensive. Some might argue that asking a software developer to write code from some random pre-baked challenge is a valid measure of both skill and pressure-coping abilities. I vehemently disagree. Although programming superstars are to be lauded and highly compensated, in the final analysis, software development is a team sport. The Internet was invented to allow developers to share code and algorithms; websites were an afterthought. The entire concept of open source software presumes access to code for reference and innovation. One might as well employ drunk coding for selection criteria. Or coding while being chased by a barrel of hostile monkeys.

Any organization that would ask a software developer to write code on command without allowing said developer the means and latitude to satisfy the development requirements using references, sources, and tools normally used in software development is like asking a basketball players to try out for a team with no ball, no basket, no hardwood court and no team with whom to interact. “Let me see you dribble without a ball.” It is akin to asking a musician to explain the tools of music composition while composing nothing.

Such a selection process for any sports team or music ensemble would be absurd and pointless. In my view, the same is true of any organization or manager requiring the passing of a real-time coding exam from a potential hire. And yet, there’s an entire industry based on coding challenges as such.

So today I am writing to complain about the absurdity of coding challenges, the litany of code monkeys who inevitably line up, to unquestioningly respond to such challenges, and the advent of code generation from natural language processing (NLP) innovations, which will replace many of those code-challenge monkeys in short order. If you hire software developers based on time-based code challenges wherein you observe each key stroke and don’t provide interaction, discussion or reference possibilities to outside sources during the process you are operating with the mistaken belief that a developer who does very well under such circumstances knows how to code. The fact is, a LITANY of search results from a Google query on the topic should be sufficient to inform if not dissuade one from such an approach — if you can easily find the solution to your coding test via a Google search, chances are, so can your potential new hire. What does that encourage? The successful code monkey must learn quite well by rote. And if that is what your organization lauds, then you won’t need those code monkeys for long if at all.

With the number of software developers worldwide tripling between 2010 and 2030 it stands to reason there is a growing queue of coders ready to stand in line for job opportunities. Global human population in 2030 is projected to be roughly 23% greater than it was in 2010. So it also stands to reason that the tripling of the number of software developers over the same period means that a steadily increasing greater share of human brain power is employed in the various tasks around software development. Thus, the rise of the code monkey is also to be expected. Worse than the code monkey is the keeper of a barrel of code monkeys, if the keeper actually views those monkeys as such. How can one tell such a keeper? Requiring the code-on-command test is a really good clue.

But as with so many ML/AI use cases popping up these days, the code-on-command test too will soon fall by the wayside, overtaken by more sensible approaches to probing for technical expertise, or at least the ability to obtain technical expertise. And what of those managers who obstinately continue to employ such tests? Obviously market forces will reduce those so inclined to the dinosaur ashes they were always intended to become; the unwavering power and force of ephemeralization will see to that.

Code ex machina

To bolster my claim that the code-on-command test to determine level of technical expertise is one of the primary destructive forces in the known universe, I point to advances in NLP for support and suggest that code generation by ML/AI models will soon supplant the jungle of code monkeys currently in training. So I suggest an experiment: Find a few of those code challenge questions and ask a machine for a solution.

For this experiment, from a field of choices, I picked one rather easily deployed application to generate python code. Python, as you may know, is one of the top choices for ML/AI programming languages and has even eclipsed Java in popularity in recent months. The Cogram plugin for Jupyter Notebook seemed to be the ideal target. After an hour or so of configuration futzing, I did manage to get a notebook fired up with the code completion extension on display.

The Toggle switch is important. If set to the ‘on’ or open position, hitting return after a comment in Python will start the code generation process, which you may not want. The default position is open. So close it on a fresh notebook until you want code. Enter your comment(s), and then set the Cogram Toggle to open, which allows you to then see the Configuration icon as well as the Operating icon. You can configure Creativity and the number of Suggestions or code snippets to generate.

From the HELP inline documentation:

👉 Cogram works best with some context. In an empty Jupyter Notebook, start out with one or two cells where you write code manually, then begin using Cogram.
💡 You can generate code by typing a comment, then waiting for Cogram’s completion:

👉 You can cycle through different code suggestions with the arrow keys.

The hypothesis for my quick experiment is simple: Modern code generation by machine can match the output of humans forced to endure time-boxed coding tests during interviews.

In all fairness, my quick experiment was not meant to be a rigorous determination to prove or disprove the hypothesis, but rather an interesting set of activities from which to derive a blog of complaint to be filed with all the code monkey keepers on the planet. What I discovered during this set of activities was the fine ability of platforms like Cogram to augment programming productivity. I will want to use Cogram going forward as much as I would want other required tools. I think innovations like those Cogram can monetize will be as important as some of the more useful libraries and packages we take for granted today.

Here is but one bit of code generated by the Cogram extension on my notebook:

# Print a fibonacci series using recursion.

for i in range(n):
print(fibonacci(i))

Bear in mind, the ONLY line of code I wrote was the comment and then I hit return. Cogram did the rest. Also bear in mind this specific question was taken from the long list of typical coding questions asked in the coding tests I find to be so objectionable. So listen up, code monkeys and keepers: the machines are coming for your jobs. You’ve been warned.

Now, one might ask, “If not coding tests, how then can we determine the coding abilities of a potential hire?” And that’s a fair question. For which I ask a simple question in response: “What specifically are we aiming to measure with such tests?”

Are we looking for the most Pythonic coders? Is that really what we need? Do we value rote memorization skills? Walking Wikipedias? Or is it something else? After all, programming languages come and go. Just a few years ago it was Java. And before Java it was C/C++. And now it’s Python. Tomorrow may very well find Javascript or Go or Julia to be the most in demand. Do we want SMEs in nuances and corner-cases of those specific languages? Or do we want monkeys who can think critically, learn easily, and express solutions using abstract logic and pseudo code?

Another question I would ask is also very simple: In an age when no-code/low-code platforms are receiving the lion’s share of investment, why would we insist on hiring code monkeys? Wouldn’t we be better of with at least a balance of no-code monkeys?

Think Outside the Language

Python is awesome. But it’s not the last stop on this journey. As the rate of technological innovation increases, and the engines of data generation continue exponential bursts of more and more and more, there must be the emergence of new and more succinct and more expressive ideas to reduce human cognitive load. Nowhere is it written that code must be in any language at all. If nothing else the emergence of code generation by machine should teach us that. And coming from NLP? Were we looking for code generation? No! We were looking for some understanding by machine of written human text. Take us a step closer to passing the Turing Test perhaps, but code generation? No! That happy by-product of modern NLP is but one small consequence that was not, to my thinking, intended when when the first steps toward NLP were suggested over a hundred years ago.

I would implore the monkey-house keepers to rethink the use of coding tests and I would encourage the code monkeys to rebel — throw off the chains of time-boxed coding tests. Just say no. Explain why such tests are counter-productive. Think outside the language and emphasize the innate ability we all share to quickly learn and adapt. Isn’t that ability what sets us monkeys apart in the first place?

Please see my github repo for the Jupyter Notebook from this fun experiment. And if you do code and have yet to experiment with Cogram, I strongly suggest kicking those tires.

“‘Come now, and let us reason together,’
Says the Lord,
‘Though your sins are like scarlet,
They shall be as white as snow;
Though they are red like crimson,
They shall be as wool’” (Isaiah 1:18)

In part 1 and part 2 of this series I hope I explained with a modicum of clarity that NLP is hard — in fact, NP-Hard, which is to say we probably do not yet have the means nor materials to get there. What does it mean to ‘get there’? Per Turing, when a machine cannot be distinguished from a human in an open-ended conversation, we might then presume that the machine is ‘thinking’ simply because we have no other mechanism to measure ‘thinking’ otherwise. Three questions come to mind:

1. Can we reliably measure thinking in some other way besides conversation?
2. If a machine does ‘think’ does that imply machine consciousness?
3. Is thinking required in order for ‘reason’ to emerge?

Measuring Thinking

Human brain activity is routinely measured via fMRI; tools sensitive enough to record the activity of a single neuron. https://engineering.mit.edu/engage/ask-an-engineer/how-are-thoughts-measured/

Neuron activity does not necessarily correlate to ‘thinking’ however. Your heart beats whether you are aware of it or not. Does that sort of nervous system activity mean ‘thought’ or thinking is involved? If so, then doesn’t a computer program that manages some real-time process also qualify? If so, then doesn’t any insilico neural network also qualify? If not, then perhaps neuron activity alone is not sufficient to conclude that a human (or machine) is actually thinking, and therefore, other measures must at least be included.

What does it mean to think? Per Turing once more, from his seminal paper (FIRST PARAGRAPH):

I propose to consider the question, ‘Can machines think?’ This should
begin with definitions of the meaning of the terms ‘machine’ and
‘think’. The definitions might be framed so as to reflect so far as
possible the normal use of the words, but this attitude is dangerous. If
the meaning of the words ‘machine’ and ‘think’ are to be found by
examining how they are commonly used it is difficult to escape the
conclusion that the meaning and the answer to the question, ‘Can
machines think?’ is to be sought in a statistical survey such as a Gallup
poll. But this is absurd.

Brain activity alone cannot be the measure of thinking. Yes, there are clearly strong correlations between thought and measurable brain activity. But the cause? Does brain activity ’cause’ thinking? Or is it the other way around? Or is brain activity Plato’s shadows on the cave wall as it were, with consciousness being the source of light? I would argue that we are still swimming in very shallow waters when it comes to a science-based (i.e.: neurological) understanding of consciousness. When we have evidence for human perception absent of brain activity in near death experiences, and no tractable explanation from reductionist approaches, I believe it is safe to say there is so much we just don’t know. Rather than delve further into that topic, when it comes to consciousness, let me say that we simply do not yet have a satisfactory explanation from science for the emergence nor nature of human consciousness. Theories, yes. But science is still watching shadows dance on the wall.

Where does that leave us with respect to measuring thinking? Turing thought it through pretty well, in my view. Short of joining the Society of Solipsists on Facebook, which is absurd, how can we know thought exists outside my own internal universe? Which brings us back to a belief: I can start to presume you think too if we satisfactorily engage in conversation of some kind. Why? Because language is infinite in possibilities. Yes, I can and do predict what you might say in response to something I might say. But more often than not, I am surprised by what you say, and whether we agree, disagree, entertain, or confound each other, we are engaged in a process that, by it’s very nature, is delicately poised on an infinite adjacent possible.

While we can produce applications that do appear to exhibit some level o ‘understanding’ of human language, no system to date has truly passed Turing’s test. None. Despite decades of research, incredible Moore’s Law fueled super computers, application specific chips, trillions of gigabytes o data, and well-funded teams of researchers all around the world, we have yet to pass with clarity and consistency Turing’s (simple?) test. Can we reliably, consistently, and accurately measure ‘thinking’ in some way other than conversation? No. We cannot. We can measure, to some extent, use case-specific language abilities. But nothing in the way of general conversational skills when it comes to machines.

Thinking vs Consciousness

Is there a difference between thinking and consciousness? Can one be conscious without thinking? Can one think without being conscious? Are they synonyms? Depending on academic perspective, the specific answers to those questions might take on different tones but the colors would likely be similar, whether neurological, philosophical, psychological or spiritual. Thinking and consciousness are not the same thing. Consciousness is a state of being. The car is running and the engine is idling but nothing is moving until we put it in gear and hit the gas peddle. Thinking is what happens when we start moving. Energy is expended and a series of actions (thoughts) occur or are allowed to occur. Although some may take issue with this wording, I think it is fair to suggest, based on evidence, that one can be conscious or aware of existing but not in a thinking state. Meditation techniques routinely encourage practitioners to achieve such pure, albeit difficult, states of consciousness. We can measure brain waves that tend to correlate with categorical states of consciousness, some of which imply a type of thinking. Perhaps would could do the same with machines? Would that then mean the machine is ‘thinking’?

Some noted philosophers would argue that conscious thoughts are an illusion. That’s not to say necessarily that consciousness is an illusion, but the idea of being conscious of a thought is an illusion. Others would argue that consciousness itself is an illusion. (https://mindmatters.ai/2019/01/has-science-shown-that-consciousness-is-only-an-illusion/) Given the unsettled and often unsettling debates regarding the nature of human consciousness and thought as a function of that specific thing, illusion or not, it should be clear to any thinking person that the experience of consciousness and the experience of thinking can be quite distinct, although very much related. One (consciousness) contains the other (thinking) and not the other way around. Mind gives rise to brain. Mind can change brain (neuroplasticity).

For humans, thinking implies consciousness. Might we say the same for machines? If so then the question Turing posed: “Can machines think?”, must also therefore imply, “Can a machine be conscious?” And we continue to be faced with the solipsistically unsolvable question: How can we ever know?

Whence Reason

To the third question above, is thinking required for ‘reason’ to emerge? Clearly it depends on what we mean by ‘reason.’

Depending on the definition, ‘reason’ implies conscious(ness) application of logic. In other words, thinking. Yes we can produce logic-based applications that blindly apply rules to given inputs. And we can produce networks of simple logical mechanisms named for what we believe are the basic function of human neurons. Such networks also learn and then apply rules to a given input. Strictly speaking, is that reasoning? If we leave thinking out of the definition, is it accurate to say that output based on the application of logical rules to a given input is ‘reason’? If so, then isn’t a mechanical relay machine as proposed 70 years ago a reasoning device? Philosophically, reason might be reduced to the process of drawing logical inferences, in which case our machines are becoming increasingly more reasonable, are they not?

If reason emerges from machines, or at least a level of reason such as to actually give rise to an illusion of consciousness, answering an emphatic ‘yes’ to Turing’s original question, then perhaps NLP is not the answer. At least not NLP alone.

A relatively recent blog entry from DeepMind of Google discussing Open-Ended Learning Leads to Generally Capable Agents, points to some interesting possibilities. The upshot of the research is promising insofar as AGI may be concerned, which could also bring us asymptotically closer to passing the Turing Test hurdle. Imagine a game space, some 3-d environment in which autonomous agents learn and compete or game-space dominance. Borrowing on lessons learned from other game solving approaches such a Q-Learning, populations of agents are trained sequentially, with a Darwinian seasoning: each new generation of agents capturing from the best agent in the previous generation, iteratively improves the frontier of normalized score percentiles, while at the same time redefining the evaluation metric itself – an open-ended learning process. To me, reading this blog in 2021, I was reminded of many of my own musings, published in 2004, regarding autonomous agents, Darwinian selection processes, and the evolution of intelligence insilico. If you haven’t read my book, it may yet be of some interest as the Network Age continues to unfold.

If general capabilities can and do emerge in such a fitscape, as described by DeepMind, then it seems to me that NLP, married to agents in such an environment, perhaps even married to hybrid systems such as Marcus described, might actually allow Pinocchio to at least discuss matters with us, allowing us to suspend disbelief long enough to accept Turing’s wisdom and the test he provided so many years ago.

So here today I declare there is yet hope. These are dangerous times. Despite disturbing evidence to the contrary in recent years, I still believe there is yet hope for humanity. And software may yet be tamed to work with us lowly humans rather than controlled by nefarious global entities to enslave us.

A final word on Turing for this series. Assembling notes from several students who attended Wittgenstein’s Lectures on the Foundations of Mathematics at Cambridge in 1939, Cora Diamond of UVA gave us a glimpse into a special moment in the 20th century, when the gifted student Alan Turing studied and debated with the renowned philosopher, who presumably influenced Turing’s views regarding both language and mathematics.

Human language has infinite possibilities — the infinite adjacent possible. Words alone are symbols. The meaning of words provide the essence of language understanding. Humans have experience, culture, interactions, cycles, seasons, and a mortal coil from which to breathe. Machines do not. Meaning comes from our human lives, not inherent in the language. Per Wittgenstein, “For a large class of cases – though not for all – in which we employ the word meaning it can be defined thus: the meaning of a word is its use in the language.” From this small bit of insight, perhaps the path to Turing’s ultimate test was paved.

Let us reason together. We can still do this.

“‘Come now, and let us reason together,’
Says the Lord,
‘Though your sins are like scarlet,
They shall be as white as snow;
Though they are red like crimson,
They shall be as wool’” (Isaiah 1:18)

In part one of this series I made the assertion that NLP is hard. We might even say it is NP-hard, which is to say that in theory, any model we would hope to engineer to actually ‘solve’ the context-bound language of human beings is at least as hard a mountain to climb as is the halting problem for any given (sufficiently complex) program. For an analysis for linguistics from the perspective of machine learning, Ajay Patel has a fine article exploring the matter.

Although we have witnessed incredible advances in machine intelligence since Turing first asked, “Can Machines Think?“, we are still pondering exactly what it means to ‘think’ as a human being. The field of consciousness studies has produced laudable theories, to be sure. But we don’t yet know — we do not yet agree on — what precisely we mean by ‘consciousness,’ let alone share a congruent theory as to what it is. From pure (eliminativism) denial to panpsychist projection to good old school dualism, we still wrestle with the mysteries entangled in the most common aspect of human existence. We know it when we experience it. But we cannot say what it is. It’s no wonder Turing ultimately threw up his hands when he realized that any existential foundation beyond solipsism required an acceptance of other as conscious. Thus, if it walks like a duck, and quacks like a duck….you know the rest.

The Sapir-Whorf Hypothesis asserts that the structure of language shapes cognition; my perception is relative to my spoken (thinking?) language. Some linguistic proponents of this view hold that language determines (and necessarily limits) cognition, whereas most experts only allow for at least the influence of mother tongue on thought. Wittgenstein poured the foundation and Sapir-Whorf laid the rails. And apologists for cultural relativism were empowered in the process.

But do I really think in English?

Have you ever known what you wanted to say but could not remember the precise word? You knew that you knew the word you wanted to say, but you could not find the specific word in your cognitive cache. Some internal mental subroutine had to be unleashed to search your memory archives; within moments the word was retrieved. But you knew you wanted to say something — some concept or idea — but the associated word did not immediately come to mind.

In 1994 Steven Pinker published The Language Instinct, in which he argues that human beings have an innate capacity for language. Like Chomsky, evidence of a universal grammar is presented. Per this view, human beings instinctually communicate, honed by evolutionary forces to succeed as hunter-gatherers. Written language is an invention. Spoken words too are inventions, to facilitate the natural instinct for structured communication. Like a spider’s instinct to weave a wab, human beings have an instinct for structured communication. If this is so, then we ought to be able to discern the rules of the universal grammar, and eventually automate the production of words such as to effectively and meaningfully facilitate communication between humans and machines. And yet, some six decades since the dawn of computational linguistics, we are still frustrated with conversational agents (chat bots). We do not yet have a machine that can pass the Turing Test, despite many attempts. And NLP is still confined to a short list of beneficial use cases.

Must we solve the hard problem of consciousness in tandem with finding the machine with communication skills that are indistinguishable from humans? Per Turing, it’s the same problem.

Neuro-symbolic Speculation

Despite impressive results from modern deep learning approaches in NLP, Turing’s challenge stands: invictus maneo. So if neural networks aren’t enough, then what? We’ve tried rules-based systems. That didn’t work. Despite a resurgence of knowledge graphs, are they not too subject to the intrinsic problems expert systems faced in the 1980s?

Enter Gary Marcus. The solution, per Marcus, is a combination of deep learning and knowledge (rules-based) AI. It’s the next thing. The revolution is here and we have a flag….maybe.

To be fair, Marcus did not invent the idea of Neuro-symbolic AI. In all probability it was IBM. In my view we never give enough credit to IBM for leading the field of machine learningin so many ways. Maybe because they’ve been around since the dawn of the 20th century and their stock pays a decent dividend. But Marcus has planted the flag of Neuro-symbolic AI firmly in the modern noosphere, and so we speculate with him here.

His basic idea is simple: take two complementary approaches to AI and combine them, with the belief the the whole is greater than the sum of the parts. When it comes to solving the hard problem of language, which appears to be directly related to consciousness (and therefore AGI) itself, we must stop hoping for magic and settle for more robust solutions for NLP-specific use cases. Our systems should being with basic attributes: reliable and explainable. That seems like a logical step forward. And maybe it is.

Marcus has suggested a 4-step approach to achieving this more robust AI:

1. Initial development of hybrid neuro-symbolic architectures (deep learning married with knowledge databases)
2. Construction of rich, partly-innate cognitive frameworks and large-scale knowledge databases
3. Further development of tools for abstract reasoning over such frameworks
4. Sophisticated mechanisms for the representation and induction of cognitive models

Sounds reasonable, doesn’t it? And perhaps the Marcus recipe will incrementally improve NLP over the next decade to the point where chat bots at least become less annoying and model review committees at Global 2000 firms will have ethical ground cover when it comes to justifying and rationalizing predictive results, regardless of how unsavory or politically incorrect they may be.

But Turing Test worthy? Maybe not.

In part 3 we will dive a bit deeper; it will be rewarding.

“‘Come now, and let us reason together,’
Says the Lord,
‘Though your sins are like scarlet,
They shall be as white as snow;
Though they are red like crimson,
They shall be as wool’” (Isaiah 1:18)

Ten years ago I made the decision to fully restore my hands-on coding skills, spend more of my cycles on learning and coding and less on management, consulting, and management consulting. It was one of the best career decisions I ever made.

During most of that decade my focus has been on all things AI, the most recent three years of which have been devoted to Natural Language Processing (NLP) on behalf of two different firms — one very large and one very small. Prior to those engagements my pursuit of data science skills and machine learning knowledge was more general.

For the sake of this collection of blog entries, please consider the thesis of Thinking, Fast and Slow; we humans use two complimentary systems of thought:

– FAST, which operates automatically. It is involuntary, intuitive, effortless
– SLOW, which solves problems, reasons, computes, concentrates and gathers evidence

Andrew NG, one of the pioneers of modern AI, is famously quoted as saying,

“If a typical person can do a mental task with less than one second of thought, we can probably automate it using AI either now or in the near future.” — Andrew Ng

If Ng is correct, then the probability of creating a Machine Learning model to meaningfully process (i.e.: extract meaning from) the image in the example above is far greater than the probability of finding similar utility with an NLP model to process the associated text.

As the adage goes, “A picture is worth a thousand word.” But those words do not adequately convey the inverse of the relationship. How many words does it take to paint a picture? And how do we process those words with a machine? More importantly, what do those words mean?

Why is NLP so hard?

Recent advances in NLP have made headlines, to be sure. From BERT to ERNIE to GPT-3 and all the variants in between, the promise of Turing Test worthy models seems to be just around the proverbial corner. But we are still not there. Not even with 500 billion tokens, 96 layers and at a cost of $12 million for a single training run.

In my view, when it comes to Machine Learning and Artificial Intelligence, NLP is far more difficult for machines to get right than any other facet of AI, and it has very much to do with the three characteristics listed below and the fact the when humans process words, whether written or spoken, both FAST and SLOW thinking are involved. And more.

My focus on NLP over the past three years has led me to a few general assertions regarding human language, which I now share:

1. Human language has infinite possibilities. Regardless of the language or vocabulary, an infinite number of sentences are possible.
2. Text communication alone is multidimensional and can be inherently ambiguous.
3. Human communication is comprised of far more than words.

Evidence supporting the assertions above are rather trivial to gather so I make no effort here to prove or validate the above. Please take issue with these as you see fit. I would be pleased to learn if and why any of these assertions are somehow invalid. Assertion #1 above may be debatable from a computability perspective, but for the sake of these blog entries, let’s just go with it and not get side-tracked.

So…NLP is hard. Really hard. That’s not to say we can’t make progress with use-case specific advances. Leaving the Turing Test out, what can we do? And how can we improve? This series of blog entries is devoted to the exploration of hybrid approaches to solving some of the more difficult problems machine learning enthusiasts face when processing human language. Today a handful of NLP use cases are business worthy. Application of NLP can be found in a range of business contexts, including e-commerce, healthcare and advertising. Today, the following are the top NLP use cases in business:

• Machine Translation: Frequently used NLP applications, machine translation enables automatic translation from one language to another (eg: English to Spanish) with no need for human involvement.
• Social Media Monitoring: Controversial at best, increasingly used to detect and censor viewpoints that may not conform to ‘factual’ analysis.
• Sentiment Analysis: Helping businesses detect negative reviews of products and service.
• Chatbots and Virtual Assistants: Automate question/answer as much a possible to make human customer assistance much more productive.
• Text analysis: Name/Entity Recognition (NER) is one example of text analysis use cases.
• Speech Recognition: Alexa, Siri, Google, Cortana and others.
• Text Extraction: Search, but smarter.
• Autocorrect: Spell Check and more

These general buckets of NLP use cases have seen NLP become baked-in components for a growing list of applications and services. While the utility of these functions grows daily, so does frustration levels (who among us has not wanted to throw a smart phone against the wall when speaking with an impersonal, clumsy chat bot?).

It is clear that NLP in practice is here to stay. But even with (unwarranted) Turing-Test-passing declarations made by otherwise intelligent humans for innovations such as Google Duplex, it is clear that the use-case specific advances we tout are not the AGI Turing was looking for at all.

Consider the context for Turing’s recommendation of the Turing Test to begin with: Can machines think? How can we know when a machine is intelligent juxtaposed to human intelligence? A programmable assistant to make calls on my behalf, regardless of how human-like it may sound, is in no way capable of handling spontaneous interactions easily handled by us flesh-and-blood folk. Our human scope of spontaneity is infinite, whereas the bot is not. At least not yet.

So what are we to do? When it comes to NLP what comes next? I will examine a few real possibilities in subsequent entries.