Yet Another list of Really hard Problems, Episode 0

No Comments

OK, I admit that probably I'm just stupid and do not understand the basics that others understand, but I write it out here anyway with a hope that may be someone will send me a comment by e-mail. The thing that as of the writing of this article I'm not capable of wrapping my mind around is that how is it that it is not possible to finance an activity, where an extremely costly software development problem that is often used is taken and solved by taking the problem, putting some thought into it and releasing the software after it is READY, AFTER THE PROBLEM IS SOLVED, and not by some mad deadline?

Examples of such problems, some of them solved by now or in progress, are, in no particular order:

  • Walk a tree recursively, breath first and depth first and do it by avoiding branches according to a given function and do it regardless of data structure, regardless of whether it is a graph database, file system, set of nodes in RAM, etc.
  • Implement a GUI that uses a proper GUI system that does not depend on the stack-of-cards model (example: most web applications) and does not require a lot of boilerplate (example: Qt) and does not require a lot of basics to be written manually (example: Java GUI libraries) and is actually executable on all modern and popular operating systems.
  • Have a proper packaging system that treats different versions of a software as different packages and allows the packages to be installed in a predetermined order. (The Nix package manager is a very recent phenomena. The people, who designed the RPM and DEB package systems clearly did not give much thought on the software development perspective of a package management system, resulting the widely popular mess that currently exists.)
  • Have an applications programming language that uses whole numbers and rational numbers as the kind of numbers that are known from mathematics in stead of bringing the exception upon exception of the CPU-supported ints and floats to the application level? (Control over number types is necessary at some cases, but the use of the CPU-supported-numbers should be a choice, not a requirement. Luckily the people behind Ruby saved the "day".)
  • Have an operating system that has a decent security model, runs on cheap-enough hardware and has a decent driver API. (As of 2015_11 the Genode OS is not yet ready, but it is moving to that direction and it has some demos published.)

The financing of that work does not seem to fit to academia, universities, because they need proofs of concepts for scientific articles and if they offer a proper, finished, software product, then their competitors from other universities will outperform them at the scientific articles front.

The financing of that work does not seem to fit to industry, start-ups and mega-corps alike, because the businesses operate on short-term bases and for them an automation system that offloads 30% of the work from expensive humans is already a financial gain and if the 30% savings during the period that it would take to get the system completed are greater than the cost of the incomplete system, then they have a financial incentive to use the incomplete system. Even if long-term goals are declared at the top of the royal hierarchy, daily operations revert to short-term perspective, because the various levels of middle-managers need to show something to justify their existence during the time, when the proper system is being developed. The next iteration can also be some incomplete system that does everything that the current systems do, but offers some (X<100)% of work offloading from more expensive options. The result is that due to the fact that there is never a financial incentive to do a technically proper job, none of the systems that are developed only for financial gains ever get properly designed and completed. If some cost item becomes prohibitive to the survival of some industry, the business people just switch from one industry to another, find other ways to earn money in stead of investing in technology development that would save the industry. Often times paying lobbyists and lawyers is seen as a cheaper and more efficient option than paying engineers and scientists (A few examples: Microsoft and software patents, various genetically modified organism sales corporations, chemical industry, fossil fuels industry).

The financing of that work does not seem to fit to various money redistribution foundations, "charity" funds, either, because they depend on propaganda to survive. Their managers need to show the founding billionaires staggering propaganda spectacles to justify their activity as foundation managers or the foundation depends on public donations, which in turn depends on successful propaganda campaigns. Anything really technical and serious fails the propaganda requirement miserably or the non-technical managers just get ripped off by crooks, who promise Research and Development, but in stead of sincerely failing or pursuing discovered paths that were not part of the project proposal, as is normal for R&D, they just leave the R&D development work undone and lie that they failed. That is to say, the "charity" foundations do not have a chance even, if they were willing to take huge risks and were willing to invest in projects that have low propaganda value.

How about the army? Regardless of the set of gangsters that the army works for. The armies are not an option, because an army is of any good only as long as it can beat other gangsters' armies in at least some circumstances. Therefore, given that secrets can be dug up by spies, any R&D for the army must cover only subjects that the given army has exclusive opportunities of leveraging. Otherwise the R&D only increases the technical level of warfare, imposes investment requirements to all armies in the given world. The army that has the greatest budget, can use its budget as the exclusive opportunity, and therefore should have the motivation to increase the technical requirements for all its competing armies. That's an interesting solution, because in that case the army, who finances the R&D, is motivated to make sure that the results of the R&D are PUBLIC, known to their competing armies, foes. On the other hand, in that situation the competing armies have a strong incentive to kill the developers and scientists, who carry out the R&D.

What about supermafia/state civilian grants? A major university can get some money, if it has proper political connections, but the rest, open source development community, Linus Torvalds at his early years of Linux, etc. are excluded. Yet, wellbeing of humanity depends on the availability of advanced products, which in turn depends on the development of those products and the cost of the use of those products. For example, electricity costs of computer hardware, fuel cost of various engines, depends on how complete their design is. A "minimal viable product" as a car engine probably has pretty bad fuel consumption or other problems that eliminate many use cases for cars. A "minimum viable product" as an operating system is the Windows operating system with all its security related problems. A "minimum viable product" of web applications is the JavaEE ecosystem with the layers-upon-layers-upon-layers of shoddily designed bloatware to get even the most basic things done. A "minimum viable product" in the form of a programming language is the PHP programming language, with horrible syntax and the duct-taped set of features that are for granted at programming languages that were actually designed, not just slammed together to quickly alleviate the pains that were imposed by the shoddy design of the CGI. And the list goes on. Yet, Linus Torvalds did not have to publish his initial work for a deadline, the Linux kernel has evolved at its own pace and the result is so good that as of 2015 the winnings of it cover mobile phones, touchscreen based document readers, scientific computing super-computers, entertainment industry, so on and so forth.

As of 2015_11 my conclusion is that true work that really raises the technical abilities of humanity can not be financed by any scheme, where the service provider has to prove to its/his client that something useful got produced, any useful services were provided. Therefore the best development work is unmarketable. One way to look at it is, would the statues and sealing paintings in Italy be the spectacles that they are, if their artists had just handed them in by some deadline? What about the Mona Lisa? Are there any examples, where something truly useful or great and groundbreaking has been developed by deadline? (The Wright brothers developed planes out of fanatism and the government financed, only-for-money, project that got all the press attention, failed. The use of radio waves was not developed for any deadline, the developers of 2. World War radars probably just got lucky to complete their project before the atom bomb developers completed theirs.) I guess that it is necessary to help others, including helping strangers and semi-strangers for money, because none of us can ever learn everything and have the capability to do everything, but machines should do the dumb work and in the future probably every surgeon has his own personal robot that does the stitches according to the personal preferences of its owner, leaving humans only the creative work. Yet, the technology to automate things works properly only, when it is completed, not assembled as some "minimum viable product". A self-driving car can not be considered to be self-driving, if in some corner cases it is left for the human driver to push the breaks, essentially requireing the human driver to still waste all of his/her time and attention on looking for the corner cases, where the machine does not do its job.

So, where does all that leave me? As of 2015_11 I do not know. I will certainly feel truly depressed, if I have to start a project with a knowing, that to be financially viable my goal is to produce "minimum viable product" crap-ware. I got lucky to have had an opportunity to work in a team, where crap-ware was not produced, but that team served scientists, not business people, and there was one, single, product that was released at our own pace and the sales argument was that in its niche it really was the best in the world. Yet, the business was "bad enough" that eventually the original founders of the company sold the whole company to a megacorp for "penuts", less than 50 million euros. All of that after years of development and after the products became mature. To me it seems that essentially that case also supports the claim that business and proper development just do not fit together. As I described earlier, the academia is no better, because it has its rush with scientific articles, regardless of who funds them. With math the solution goes from mathematician to a client and money comes from the client to the mathematician. With software, the only viable option seems to be that broken crapware that passes acceptance tests, goes to the client, money comes to the software developers or if the developers are slave-minded, to some business school graduate slave master, and after a little while the client wonders, why the software corrupts data, why the software is slow as hell and why any modification to the software costs a fortune.

Now, where am I mistaken? (I believe that sales deserves 50% of the project price money, but by the "business school graduate slave master" I did not designate sales people.) As of 2015_11 I just declare the task of selling proper development services for money as a task that is not proven to be unsolvable, but like many problems in math and computer science, has a suspicion of lacking a solution.(Example problems might be the factorization problem and the question, whether there exists any algorithms that are not executable on a Turing machine.)

This article will be improved and modified at will.

Other Similar Articles

Comments are closed for this post