First, a bit of context

The proof of concept for the ASTRID-system, a Cognitive Architecture that is based on symbolic reasoning and capable of full unsupervised transfer-learning, was initially developed in PHP. Now, you might say that PHP would not be the first choice to build such a system, and you would be right. When you look around, you won't find any recommendations for PHP in the Artificial Intelligence domain.

However, as the ASTRID-system is based on symbolic reasoning instead of statistical pattern matching (like e.g., Deep Learning), the choice of programming language didn't seem to matter that much. There were, however, valid reasons to go with PHP at that point.

The most important reason was the fact that we had access to a powerful development platform that was optimized for rapid prototyping, developing and maintaining PHP-based applications. It was logical to use this platform, as it would cost a lot less to develop the proof of concept this way.

A secondary, but also essential, reason was that PHP is quite useful for prototyping technology. Any dynamically typed and interpreted 'scripting' language would be a good choice for this phase in development. But having a PHP-focussed development platform available made the choice easy.

Selecting the right programming language

Although the initial proof of concept was built in PHP, it was clear from the start that the system would eventually be ported to another programming language. As stated before, PHP is great for prototyping code, but it is also rather slow (for several reasons). The proof of concept needed to prove that it is possible to build a symbolic-based cognitive architecture that is capable of continuous self-learning. Speed was not a primary concern at this stage, and we were confident we could address that at a later stage.

At the start of the project, it was expected that we would eventually move to Python, as that language was already prevalent in data-science and Artificial Intelligence research, particularly in Deep Learning. So, initially, we focussed on Python and researched the possibilities of this first choice, but we soon found that this would not do much better than PHP, for our technology. So, we started a thorough selection process to evaluate other possibilities.

The primary criteria were obvious. Fast execution to get to commercially viable reasoning speeds, so interpreted languages were out. It should also build on a proven and widely available infrastructure, so we didn't have to invent the wheel to get it running on any hardware or Operating System (information systems, robotics, self-driving cars, etc.). For the same reason, there should be a large and stable ecosystem of supporting technologies, like libraries and frameworks. And finally, it should cater for easy implementation of parallel processes, as the ASTRID-architecture could benefit immensely from that. These few criteria translated roughly to 'running on the Java Virtual Machine' for speed and platform robustness, and 'functional programming' together with 'immutability' to cater for parallel processing. This narrowed the options down to Scala, Kotlin, and Clojure.

Switching to Clojure

Scala and Kotlin both do 'functional programming', but they also support other coding paradigms, like Procedural and Object-Oriented. They also don't implement immutability as a requirement, but only as an option. Clojure is the only language (on the JVM) that supports the functional paradigm exclusively and uses immutability by default. So, we decided to go with Clojure. And let's not forget, REPL-driven development seemed really appealing.

Moving from PHP to Clojure might seem a big jump. Going from imperative to functional programming does indeed come with a steep learning curve. And with Clojure being a LISP-based language, that brings yet another learning curve. Finally, working with immutable variables also needs a 'reset' about how you think about software development. All these things together meant that there was some significant effort involved in switching to Clojure.

In reality, the switch went rather painless. Our development platform implements a deeply modular approach for constructing applications, and the functional model of building software just strengthened this. We were also used to an array-centric coding model, where we continually try to build functionality with arrays first. This made the LISP-way of doing things just feel natural. And immutability didn't bother us that much because we basically never reused variables in PHP either, and stayed away from global vars as much as possible. Although updating values in (immutable) variables did throw us off a bit (and still sometimes does).

Coming full circle

The one thing we didn't see coming is how important it is for us (now) that Clojure is a LISP. The LISP language was developed by John McCarthy, and we actually have a picture of him right here on our website because he is also one of the 'fathers' of Symbolic Artificial Intelligence. He developed the LISP-language to build symbolic reasoning systems, that was his primary goal. This means that the LISP-language is optimally geared towards building such systems.

Our ASTRID-system is a Symbolic Artificial Intelligence architecture, so a LISP-language should be great for building such a system. We never could have imagined what this really means. It means that, in some cases, up to 80% of our initial PHP-codebase was implementing functionality that is integral to a LISP. This resulted in more than 80% reduction in Lines of Code. One core-module in the ASTRID-system went from 1800 lines of PHP code to only 27 lines of Clojure code. Besides that fact that Clojure is inherently faster than PHP, this crazy amounts of reduction in code have a very positive impact on processing as well. It also shows there is some merit in Greenspun's tenth rule.

So in the end, we came full circle, back to the origins of both Symbolic Artificial Intelligence and AI-focussed programming languages. Both originated from the same person, John McCarthy, and it illustrates how important such a historical perspective is. We almost missed this incredible opportunity, but a thorough selection process got us there in the end. If we had added LISP as a requirement from the start, choosing Clojure would have been obvious right away.

Visit Clojure.org to read more about the Clojure programming language.

About the author:

Hans Peter is the founder and CEO of MIND|CONSTRUCT. He has experience with more than 20 programming languages, has over 30 years of experience in IT, and broad expertise in software engineering, software quality assurance, project management and business process engineering. Besides this, Hans Peter has been a serial entrepreneur for more than 30 years now, has worked as a business coach and consultant in many projects and has worked several years as teacher in the domain of software development.