Open Source is Ceasing to be Cool
©
Dr Mark Tarver, 2016
This essay is a lightly edited
transcript of a series of posts I made to the Shen news group in the
middle of 2015. They highlighted the failure of the open source
initiative in the project, analysed the reasons and suggested a
successful formula for moving the project forward. The posts led
to the formation of the much more dynamic closed source Shen Professional.
The lessons here are widely applicable to many open source projects and
developers may find the material interesting.
Evaluating the Open Source Initiative
I said that I would give the open source
initiative 6 months from BSD day (February 2015)before passing judgment;
but the 6 months is now well up. So it's time to look at what has been
achieved and what lessons are to be learned. Stay focused; what
follows is important.
Six months have passed since BSD day and it is time to take stock of
achievements. In that time the OS initiative has accomplished bugger
all. Am I surprised? No. But even my admittedly low expectations
were surpassed by the non-performance of this approach. I hope that
we're not going to follow the Maoists of the period of the Great Leap
Forward - who could not accept that their unworkable ideas did not work
and resorted to show trials of people to explain their failure.
Also I hope that people are not going to try to save this discredited
ideology from being led to the scaffold. Take heart, I am in fact only
executing a corpse in what I am going to say in the next messages.
If there is a value in this period, and it can be a very substantial
one, it is seeing why ideas fail, which can point to the right
direction. We are going to take the red pill. But before moving forward,
the whole open source strategy must be deconstructed.
Now I will warn you that in the next paragraphs open source is going to
be shredded. Why am I doing this? Just to
exercise an itch? No. In Taoism when you embark on energy
cultivation (qigong) you don't just put the new energy on top of what is
already there; the old stale, bad qi needs to be cleaned out.
Think of having to paint over a surface with lumpy old paint; you sand
down the surface first and then apply the new paint otherwise your new
paint job is screwed by the old one. So systematically I'm going to
remove some myths and clean out the bad stuff.
Myth #1 Mindshare = Success
Defining mindshare as popular support on chat sites and having lots of
users is an OS idea of success. If you define 'success' as such and that
is your goal, then 'mindshare is success' is a tautology. But if you've
graduated beyond this desire and you have other ideas of success - for
example livelihood - then this is one myth that needs to be busted.
We know now that though giving people stuff for free and achieving
traction is not at all synonymous with being economically successful. We
know this because we now have examples of projects that got lots of
freeloaders and still struggle for money - most notoriously
OpenSSL, but
also
OpenBSD and many others less well known. We also know that in any
project of significant size and complexity, money is needed on a regular
basis to pay for the kind of professional development and support needed
to deliver a really good product. This is the case with Shen.
The other problem in defining success as mindshare is that this creates
an almost futureproof model for failure. I can think of nothing more
likely to fuck your mind than going for mindshare. Why? Because simply
the world is a very big place full of people with very small brains who
can barely hold one idea at a time. The nature of things is that the
chances of your idea conquering the world is slight. So if you're
building mindshare into your success model you're building
disappointment, heartbreak and abandonment into your success strategy.
There is a certain character you see in the West; the sort of girl who
paints her nails, works at KFC and dreams of being selected for X-Factor
because she sang karaoke at her friend's hen night. We laugh at this
because we know that for her the chances of getting mindshare and being
the next Beyonce are slight. Everyone can dream; but when the girl goes
on to neglect her education and development because she's stuck on this
dream, then something has gone very wrong in her head. The aspirations
of the OS fanatic hooked on mindshare are often no more realistic than
the neurotic checkout girl at KFC.
Out of all the dubious achievements of open source none has been so
damaging as the general promotion of this neurotic definition of
success.
Myth #2: Open Source is Creative
This is an odd one. Saying OS is creative is what Gilbert Ryle called a
category mistake and in programming we call a type error; open source is
no more creative than people are divisible by 2 or knowledge wants to be
free. Creativity is really a feature of intelligent life of which humans
are supposed to be exemplars. We can only talk about institutions being
creative to the extent that they foster and encourage creativity.
Universities to an extent, used to do that, but they are corporate
machines today; they are not creative places to work.
OS has never really been creative in this sense for the simple reason
that mindshare philosophy dominates and the quickest way to achieve
mindshare is to copy some closed source work that has already achieved
it. Hence Linux, Gimp, Open Office etc. This is not to say that nothing
under an OS license has ever been creative; it just means that the
zeitgeist of the movement does not especially reward originality.
I predict that no tool of any kind
which too greatly amplifies the productivity of an individual will
ever be permitted to most developers. In this they shall follow in
the maximally deskilled assembly-line footsteps of their
grandparents. “They'll
time your every breath.” As for the “free software” world,
it eagerly opposes industrial dogmas in rhetoric but not at all in
practice. No concept shunned by cube farm hells has ever gained real
traction among the amateur masses.
Where Lisp Fails: at Turning
People into Fungible Cogs by Stanislav
Shen simply sticks out as
contrary to that trend - it is original and hard to understand. So
immediately we should at least begin to question the goals of
mindshare in connection with Shen. In fact the coupling of Shen with
mindshare thinking has been nothing short of tragic in its consequences
for this project. I want to get stuck into specifics next.
The Consequences of Adopting
Mindshare on the Development of Shen
I think a lot of people in the Shen group bought into mindshare and it
informed a lot of their work. Greg
wrote to me and said that there was no future unless Shen was more widely adopted. He then
nobly set himself to write an open source dissertation on Shen-YACC.
Bruno created a wiki which I attached to the Shen front page to allow
people to create their own documentation. People created the Shen reddit
- a very mixed blessing in some ways - and Twitter was used. All these
are products of a generation for whom social networking via the web and
the philosophy of mindshare are very much intertwined. Shen needed
mindshare to live and thrive.
But actually if you look at the most productive phase from 2011-2012,
the 'mindshare' consisted of a handful of people. You could have seated
us all round a table in Starbucks. Perhaps one day we need to do that.
The philosophy of mindshare is one of heartbreak because significant
mindshare is hard to achieve. Hence the people here were laying up
trouble and disappointment for themselves. But the task was made even
more monumental by very strengths of Shen; the advanced type system,
sequent calculus and the incredible multilingual potential which
collectively the software community did not and still do not understand.
So in effect a very small group of idealistic people with hardly any
resources undertook to re-educate the planet.
Giving Away Your Power
When this decision to educate the planet about Shen was taken, and I
don't say it was a conscious decision, the centre of gravity of the
project moved outwards and the energy was dissipated on a fruitless
goal. Effectively the philosophy of mindshare had redefined success.
Shen was in fact an outstanding success, almost a miracle; with the
minimum of funding, under harsh conditions, it had achieved exactly what
it set out to do and a small group of people had made it run under
nearly every major platform. But now success was redefined as mindshare.
This is the time when reddit made an appearance. I blessed the Shen
subreddit because I thought that reddit was some polite ladies bookclub
- obviously showing my ignorance of it. It is nothing of the sort of
course and some of its most vicious sites like Coontown - the reddit for
racists only recently closed - shows what is so often like. Reddit is a
shit hole; and to a lesser degree so are SlashDot, and Hacker News.
Unfortunately some members of the Shen group, imbibing the mindshare
poison, gave much power away to the opinions on reddit. The OS fanatics
relished their power to bully and shame and people used to mail me and
say 'You need to say something because XYZ has said such and such on
reddit'. I did on occasion call such people 'complete prats', but really
this was whack-a-mole because the supply of arseholes was unending.
Effectively the mindshare philosophy delivered the energy and soul of
the project to those who had reasons to dislike it. These people were
stupid and could have been dismissed as such but the mindshare poison
had so undermined resistance that people here could not fight them.
These online voices represented or claimed to represent the community -
they were the mindshare.
Having bowed to open source as they demanded, the group discovered
that these people have melted away and are not to be found. There is no
open source army on the march that Tycho hoped for. There never was. What
you had was a bunch of useless people getting their kicks out of
manipulating a small group of much cleverer people. It was simply a
power trip. The nadir was reached when two members of the open source
community crawled out from under their stone to conspire to expel Shen
from Wikipedia by announcing Shen was dead.
Taking Back the Power
When you've given away your power you generally end up energetically or
literally 'broke' and in a dead-end. The way this project felt mid-2015.
You look around and ask "How did I get here?" and as Willi did "What
now?". I've answered the first question and now I move to the second.
The short answer is:
you take back the power
This is the first step of
getting things going. You let go of the strategies and mind-stuff that
led you to the position of helplessness and being stuck. In this case we
let go of the philosophy of mindshare. Success for Shen is not
mindshare.
When we do that immediately the power returns to the centre where it
belongs; inside the project and to the people who made it happen. Since
mindshare is no longer operating, the opinions of the open source
community are no longer important. You don't have to give a shit about
them unless they have something significant to say - which, reading
reddit, is not often.
Doing this puts them in the proper perspective; rather like when you
were a kid and there was a huge creepy figure in your bedroom. You turn
on the light and the huge creepy figure is your dressing gown hanging
up. Your mind gave it power and made it into something huge. The
freetards want you to believe that they are huge, powerful and you
cannot succeed without their approval, but actually they have no power
unless we developers choose to give it to them. When you realise it is a
mind-trick induced by the mindshare pill they dwindle to their true
size.
Shen is a Powertool not Designed for
the Average Programmer
As long as people were hooked on the blue pill there was very little
chance of anything useful happening. The OS fanatics on Wikipedia talked
about Shen being dead. The smell of death around the Shen project is not
coming from Shen, it is coming from the decomposing body of the OS
philosophy based on mindshare and the faltering economics used to power
the project.
Underneath Shen is amazingly dynamic, untouched and better documented
than in TBoS 3rd edition than it has ever been. And the OS 6 month
period of total failure has had its purpose; at least people are waking
up to something that does not work and asking 'What can we do?'. People
are actually questioning the central assumptions of the OS initiative.
The failure of the OS initiative is a time of huge opportunity for Shen.
Open source philosophy is dead, last century and discredited and we
developers need to take back our power from it. But where do we go and
what do we do with that power? Well, first we must acknowledge that Shen
is itself hugely powerful, that potentially as a productivity tool it
can reap enormous benefits if rightly handled and developed, that it is
in many ways unique, but that being able to extract and lever that power
requires programming skills that most programmers do not have and never
will. Think of the skills needed to dogfight and pilot the F-16 to the
edge of the atmosphere and the skills needed to fly a family twin seater
under autopilot. That's the difference we're looking at between a Shen
programmer and the stock programmer assembling glue code and cranking
code generated from packages.
Using Shen Effectively in a
Commercial Setting
If we are looking at the commercial development of this language beyond
a hobbyist implementation then we are looking at an elite group.
I quote Paul Graham's famous essay,
Beating the Averages, by Paul
Graham. This is a must-read if you haven't read it and it is a must-must
read for Shenturians. Several of the passages apply directly to Shen.
Read it because I'm only excerpting with comments here. All I've done is
substitute 'Shen' for 'Lisp'.
The purpose of this
article is not to change anyone's mind, but to reassure people
already interested in using Shen -- people who know that Shen is a
powerful language, but worry because it isn't widely used. In a
competitive situation, that's an advantage. Shen's power is
multiplied by the fact that your competitors don't get it.
If you think of using
Shen in a startup, you shouldn't worry that it isn't widely
understood. You should hope that
it stays that way.
Bam! A precise
reversal of OS philosophy. Competitive edge requires access to
technology your competitors don't understand.
.... programming
languages are different: programming languages are not just
technology, but what programmers think in. They're half technology
and half religion. And so the median language, meaning whatever
language the median programmer uses, moves as slow as an iceberg.
Garbage collection, introduced by Lisp in about 1960, is now widely
considered to be a good thing. Runtime typing, ditto, is growing in
popularity. Lexical closures, introduced by Lisp in the early 1970s,
are now, just barely, on the radar screen. Macros, introduced by
Lisp in the mid 1960s, are still terra incognita.
Bam! Cognitive social
change is very very slow. Tying your project to it is as good as
ossification.
You will have a hard
time convincing the pointy-haired boss to let you build things in
Shen, when he has just read in the paper that some other language is
poised, like Ada was twenty years ago, to take over the world. But
if you work for a startup that doesn't have pointy-haired bosses
yet, you can, like we did, turn the Blub paradox to your advantage:
you can use technology that your competitors, glued immovably to the
median language, will never be able to match.
The OS idea that technology
can only be levered to advantage by mindshare is historically naive. The
French were annihilated at Agincourt by a technology that had very
little mindshare outside England.
Closed Source Shen Professional
Leaves Open Source Behind
In September 2015, after analysing the reasons why OS Shen was massively
underperforming, I moved my development to closed source. Since that
time the closed source Shen,
Shen Professional, has developed rapidly
with funding from a small group of professional programmers and
investors. Concurrency, compiler optimisation, large file handling, HTML
generation, and graphics have emerged or are emerging within a 6-8 month
interval. We're beginning our first commercial application - a health
app for yoga enthusiasts. Private money and market economics worked
their usual magic.
Lessons for Developers
I think there is a useful lesson for developers to learn from the Shen
project; especially when they are working at novel computing paradigms.
OS may not empower your project and indeed with a
98% failure rate is
unlikely to do so. As David MacIver very rightly remarked after learning
the weakness of OS.
Plans which are
predicated on changing the world before anyone will pay you any
money are decidedly bad plans.
Throwing in the Towel David MacIver
If you've worked in an OS
environment, you'll quickly discover that only about 2% of your users
will financially support your work. So it makes sense to concentrate on
that percentage. Open Source may be useful in establishing your
reputation, but once you are established you need to capitalise on that
reputation and be leery of fairy economics. At some point, you'll have
to think about close sourcing either code or documentation or training
because open source is not a business model.
Stuff to Read
The Problems of Open
Source was a pioneering essay I wrote back in 2009 laying out the
flaws in the OS approach. I revamped it in 2014.
Open Source is Not a Business Model
Open Source is Not A Business Model (and again)
Open Source is
Not a Business Model (and yet again)
That's three articles with the same title!
Are people starting to learn? Seems so; more of the same.
The Money in Open Source Software
Corruption and hypocrisy in the open source
movement. Read here 7 Reasons Why I do not Support
the FSF |
