Intervaarium Work Process

In stead of describing a series of events this text lays out a maze, a labyrinth, that can be crossed multiple ways. People, who prefer to be "dragged along", will probably not be satisfied with me( as a "tour guide", because I want people to be aware, what is happening and and why. Keeping people aware does not necessarily entail fast responses or intense communication. I prefer thorough communication, not frequent communication. As a matter of fact, my lack of frequent feedback can be very disturbing, because in stead of wasting people's time with empty talk I prefer to write/communicate when I have something to say and I do not have much to say during the time, when I'm still working on a project. (It seems like 2 extremes in one solution, but I like efficiency. The communication with me can be spread in time, but if it happens, then I try to make it so that it is somehow helpful to the other party or at least offers some perspectives that the other party might find helpful for assembling its own solutions/answers.)

A Subjective View of a General Lanscape

Please click the image
to see a larger version of it.

I emphasize that whenever possible, AVOID ANY CUSTOM SOFTWARE DEVELOPMENT, because software development entails the assembly of components to a non-tested arrangement and that is a total walk in the unknown even, if the components themselves are well known. Knowledge of the physical properties of a brick does not contain all the information about the properties of the houses that can be built from those bricks. Knowledge of the Periodic Table and physics does not contain all the information about the bacteria, viruses, insects that can be assembled from the chemical elements. Cancer is essentially a software flaw.

From business perspective a thing to notice is that even in software industry there exists a components supply chain. End users do use a database engine, but they use it only as a sub-component of their applications software. Car owners depend indirectly on the tools of car manufacturers and software end users depend indirectly on the tools that the software developers have. The end price of all products always depends on the tools of their manufacturers and the components within the products. Software developers, including me ( try to drive the software development cost down by relying as much on the open source tools and components as possible. Due to the development time writing everything from scratch is not an option even if clients were willing to pay for it.

Philosophical Perspective

Please click the image
to see a larger version of it.

The summary of the schematic is that what a person wants from his/her tools and other people is determined by the personality and culture of the person and that software as a tool is always optimized to meet those culture induced requirements, even, if it is not obvious that culture has anything to do with nonfunctional requirements of the software project. Another line of thought is that doing things well or solving something very complex takes time and to get the maximum benefits from that time investment the software components that took a long time to develop should be usable for a long time. Every software component is usable only as long as its dependencies are usable. Therefore, anything really good needs dependencies that are usable for a long time.

As of 2016 I do not know any truly long-term software projects that have had proper financing during the whole life-timeo of the project, regardless of whether the software has been developed by some mega-corporation like Microsoft or whether it has been developed by some nonprofit or academia that does the development from grant money or donations. As of 2016 it seems to me that only those software projects survive long term, which can be developed by fanatics that do not get paid for doing the development. Proper financing of software projects is welcome, but if there's a lot of money, then there's a threat that the software developers, even if they work for a nonprofit, cut corners to ship features faster and they do not make sure that the software project components' orthogonality is maximized, leading to a situation, where the amount of work that is needed to swap out the software project's dependencies or to change the features of the software, is too huge for a small team of hobbyists, who have to develop the project without getting paid. As no project gets proper financing continuously, the requirement of a huge amount of work to keep the software usable over longer period of time is essentially a time-bomb that destroys the project's usefulness to other long-term software projects that have it as their dependency.

Examples of projects that have stood the test of time, while getting occasional good financing: REDUCE, Vim, GCC, LLVM, the Linux, FreePascal, Perl, Parrot VM.

Examples of projects that survive only thanks to huge amount of money or have already been abandoned due to the lack of money: .NET, the Java, IronRuby, VRML(new VRML clones have emerged, but they do not offer anything really new that would not have been possible with the old VRML),

Philosophy Related Recommendations

Client candidates should refrain from demanding any specific technology and offer software developers only the problem description, because creative software developers solve the problem by using the technology that they feel most comfortable with. Even if 2 different software developers use the same technology (library, programming language, applications, etc.) then they will probably have different solutions. In corporate environments there's a custom to demand that all software developers use the same coding style, coding standards, but corporate environments are not the places, where creative people work. That holds also for software developers. Most creative software developers do not work in corporate environments or they are there temporarily, by accident, usually through some acquisition or due to stupidity that comes from the lack of life experience. For example, the creative person, who works in a corporate environment, might have graduated relatively recently and hasn't yet been burned enough to reach the conclusion that corporate environments are not for him/her.

Just like artist and writers each have their own style, so do software developers. Part of the style is reflected in coding style, for example, the way variables and functions and data structures are named. Part of the style is reflected in the mathematical, algorithmic, aspect of the solution. The list of different aspects, where software developers' styles differ, is too long to be described here, but one of the ways, how software developers differ from each other is, how they tolerate other software developers' style that is different from their own style. (Toleration differs from indifference by the fact that the tolerable thing matters and is unpleasant.) Ability to tolerate others reduces down to social skills. Other software developers can read my( version from my blog post titled, "2015_10 "the-best-that-I-know-of" Version of Software Development Methodology", but the summary of it is that I let others do whatever they want, in any programming language that they want, however they want, as long as they do it all at their own "sandbox" within the computational resource limits that were allocated to them. I like the slogan: "Design and let design." ("Live and let live.") That kind of approach, as You might have already guessed, tends to be pretty unacceptable in corporate environments. But, it is a fact that as of 2016 the only places where I have fit in on a "regular job" for some longer period of time is a team of scientists and apart from a few short jobs I've been working as a freelancer ever since. That is to say, I describe the landscape from a very biased perspective. If You enjoy working in a typical corporate environment, then You will probably not enjoy working with me. On the other hand, scientists and small business owners can find me very useful, because often their problems require considerable amount of creativity and flexibility to get it solved or, if not solved, then at least somewhat properly studied or rephrased as a totally new, different kind of, problem.

Many people believe that business ideas can be supported by working software in a matter of days, but as of 2016 I( am still, after more than 10 years of working as a software developer, too stupid to know, how it is even possible. In my view, with the very few, vary rare, exceptions, practically everything that is produced during the 48 hour events or some other similar events is of very low quality and totally useless. As of 2016 I do not imagine myself delivering anything like that to any of my clients. I did not become a software developer to produce crap in exchange for money. Like, imagine me as a small boy dreaming that when I become "big", then I'll produce crap that does not really help its users, costs a fortune and breaks down and is thrown away quickly, without much use, because there is nothing else to do with crap other than to throw it away. In my view a well done product is replaced only, when it either wears out, is meant to be a single use item like a condom or a napkin, or the initial requirements changed and the old version is unfit for the new requirements or some new technology has become available that allows the old requirements to be met in some cheaper or more highly performing manner. Well done products can also be just lost, like people lose their keys or gloves or luggage. As it turns out, the typical business software development style of "Agile Development" is recognized as software development pornography even by the people, who came up with the idea of agile software development methodology. On the other hand, that's how corporate development projects almost always tend to be seen from creative peoples' perspective, regardless of the discipline. The corporate factory line model with hierarchies is fundamentally unfit for any creative, intellectual, work.

I know that many people, specially those, who work as software developers at corporate environments, think that I'm a looser, a freelancer, someone, who does not even work at a "proper organization", telling that the "proper developers" are bad at their job. Well, opinions differ and that includes, how software should be developed and what environment is more productive for doing software development. Opinions differ even, how the word "good" is defined. For many people the "good" is a synonym for "popular", but I think differently. I also think that huge factories are not an outdated phenomenon, but the ownership model of the huge factories must certainly be changed. I think that the huge factories are a very nice and effective form of creating copies of physical goods and that their ownership can be calculated real-time by giving every person on planet Earth exactly the amount stock of the factory, as that person's work hours are percentage wise from all work hours that were spent on that factory. The initial capital investments from "venture capitalists" can be converted to work hours according to some minimum hourly wage plus a penalty (reduction of venture capitalists' work hours) that compensates for the rigged financial system. The newly created factory has a loan from the venture capitalists, with interest that matches with inflation, id est if the inflation were 0%, then the interest would be 0%, and the profit side of the venture capitalists would come from the dividends that are paid to all people, who hold stock. The loan must be paid back before any dividends are paid. Not a very nice proposal to the workers of the factory, but it is transparent, I think that also fair, and financially a lot better for the workers than the current arrangements are. It also allows fast bootstrapping of new companies, if there exists any need for such fast bootstrapping at all (archival copy). Workers of big factories might even do seasonal migration between multiple big factories without loosing any stock or investments. That should increase income security a lot while keeping costs down at all factories and all of that without reaping anybody off, because the workers themselves own the factories that have the cost benefits. From dividends point of view the 2 types of stock, the venture capitalists' stock and the workers' stock, do not differ, but to protect the more simple-minded workers, the worker's stock can never be sold and can change its ownership only through inheritance. In a way the workers' stock can be seen as a form of "citizen's salary" that encourages work and discourages slackers. To make the new ownership transition revolution free, the new business order can be gradually adopted by creating one additional company type called Calculated Ownership Company, abbreviated as CalCo. All of the new ownership related legislation is just a legislation about CalCo-s. The summary of this passage is that huge factories are fine, provided that their ownership model is upgraded, and I respect the so called "blue collar" jobs, but every job needs conditions that fit the job and in my opinion the factory style of arrangement is UNFIT FOR CREATIVE WORK and that SOFTWARE DEVELOPMENT IS CREATIVE WORK.

Technical Perspective

As I'm always trying to learn new things, become better at what I do, the technical perspective will, hopefully, change. The key phrases of my current version are: graduality between comfort and reliability (including security), VirtualBox virtual appliances, multi-programming-language applications, agent-oriented programming that uses asynchronous messaging. Application code is written in multiple programming languages, but the general scheme is that for application development and development of algorithms that are not similar to signal processing algorithms I use Ruby, for anything that requires raw speed I intend to use ParaSail, for compatibility and legacy reasons I use C/C++ and all the rest of the languages are for integration or as third party components. Web browsers are seen as a 3. party components. My own JavaScript libraries exist only to use web browsers. Desktop GUI-s will be initially done with a web browser, but in the long run there will be an additional Lazarus and Free Pascal option. The user interfaces, including GUIs, will use the RASTCLSP_t1 specification. The RASTCLSP_t1 is a summary of some of the lessons learned during the years of development and usage of the Raudrohi JavaScript Library.

Please click the image
to see a larger version of it.

The graduality between comfort and reliability is created because people do not seem to accept properly secure applications. Proper security often requires either some extra work with cryptographic keys or installing some extra software to their computer or placing and configuring some computers to their LAN. In the past, before autumn 2016, I tried to deliver just the proper, secure, version, but my clients did not like the security and reliability related hurdles, so I needed to make compromises to make my software acceptable to them. If I'm essentially going to ruin the thing anyway to make it acceptable to my clients, then I might as well just go all the way and offer 2 types of deliverables for the price of one: the proper version and the comfort-enhanced crappy version. The project price will always include the price of the proper version, I will never give any discounts on that, but clients, who think that they do not care about the proper version, may just download the crappy version and dowload the proper version years later, when they change their mind and realize that they actually do need the proper version. All of that without paying anything extra for the later download. (I know that I sound a bit grumpy here, but what can I say/write? If that's what the clients want, I can also adopt to serve them better and I did try to offer them the good stuff from the very start and I intend to always have the good stuff available for them, should at least some of them eventually figure out, why they actually need it.)

The main role of VirtualBox virtual appliances is to allow applications to be changed, updated, years after their initial release, without having to spend time/effort on setting up an old development environment that is required for their development. For example, if there is a need to change text in some menu of a very old application and the development deliverables of that old application contain a virtual appliance that has all of the development environment of that application set up, then the changing of that text is economically feasible, because all it takes is to boot up the virtual appliance, change the text, re-build, run tests and re-deploy. The main use case for the virtual appliances is to allow the project deployment deliverables to be integrated with newer software and hardware that was not available during the publication of the project deliverables. The philosophical background of the virtual appliances is that doing something well takes time, which is a substantial investment, and to make that substantial investment pay off, the software components have to be in use for a long time. The longer the software components are in use, the greater the probability that they need to be updated to make them interoperable with other, sometimes newer, components.

Some Bold Technical Predictions

Prediction 1

As of 2016 I predict that most of the web browser technology will become obsolete the moment megacorporations stop pumping money into their development, because the amount of different standards and quirks that web browsers have to support to function properly is too big for hobbyists to update. Hobbyists must be capable of updating the project, because no project will ever be properly financed all of the time, but software can be relied on only, when it is always available and software is always available only, if it is always kept up to date. The update necessity comes from the fact that hardware physically wears out and newer hardware has new specifications. The new specifications impose updates to operating systems. If operating systems get updated, then it is very likely that the operating system interfaces, libraries, also change. That in turn requires applications software to be recompiled and consequently everything that depends on the recompiled applications probabilistically has to be recompiled, including the development tools for building end user applications. Newer versions of the development tools and libraries might not have the backwards compatibility, which is where the need for manual updates arises. Eventually the update requirements reach web browsers.

The amount of code changes within an update might not be that significant, but the developers need to have a really good knowledge of the components that they update/change. If there are a lot of components to update, then the learning curves accumulate. To avoid introducing flaws/bugs, the developers have to know the components really well. The greater the learning curve, the more time it takes to cross it. Time is something that hobbyists and enthusiasts do not have, because in stead of working on their pet projects, they have to earn a living. That is why the megacorporation projects that require a lot of money/developers die the moment their financing ends. Projects that die the moment their financing ends are not available long-term. Projects that are not available long-term, are not suitable for being dependencies of projects that must be available long-term.

Prediction 2

As of 2016 I predict that chat-bots will replace most forms and some wizards. The chat-bots are an example of 2016 chat-bot technology. A chat bot does not need to interact with a web browser, which makes it possible to use chat-bots even, when the web browsers are history. Chat-bots may accept input in multiple forms, including sound and gestures.

Prediction 3

As of 2016 I predict that the future Internet will include an element of physical P2P links that allow communication to take place outside of the super-mafiosi controlled, centralized, Internet Service Provider networks. To make it harder to gather metadata, the P2P links will be temporary and random. (The 2016 arrangement, where anybody, who offers data exchange service to others, must help the Police to tap that communication, is just plain awful.)

Infrastructure Development

My custom, open source, infrastructure consists of virtual appliances, memory card images, forks, my own libraries and custom specifications that document parts of my own libraries. The centerpiece, the crown jewel, of all of my software is the Kibuvits Ruby Library, which is used as a dependency of build scripts, code generation, custom development tools. In the long run the aim is to complete the Silktorrent project, but as of 2016 autumn the initial version of Silktorrent packet creation and verification software is ready for use.

As of 2016 a totally unsolved problem, to the point that I do not even have a clue, how to solve it, is, how to deliver software to Windows and MacOS users without making them pay for some Linux/BSD hosting service. Once upon a time I thought that a Raspberry Pi memory card image solves it, but my clients did not like that. At some point I thought that may be they will accept a VirtualBox virtual appliance image, which they might run at their LAN, for free, even with Windows computers, but my clients found that to be too cumbersome. Software for mobile phones is outdated the moment the mobile phones become outdated, making huge investments of time/moeny to "mobile software" unjustified. Besides, the "mobile software" market is full of cheap amateurs, web designers, artists, various corporate teams. One of the people I know, Märt Tibar, seems to have solved that problem by running his own server and just bluntly charging a monthly fee for it. I think that his service and fees are fair, but at the era of Edward Snowden I like my clients to be autonomous, keep their projects and software running regardless of what happens to me. For slow-as-hell distributed storage one might try to use Freenet or IPFS, but that is only storage, not a fully featured execution environment.

On the other hand, VirtualBox virtual appliances are fundamentally big, because a short script has dependencies, which has dependencies, which have dependencies, which have dependencies-of-dependencies-of-dependencies-of-dependencies-of.... I think that it would be also stupid to hope that all of those dependencies-of-dependencies-of-dependencies-of-dependencies-of.... are readily available at someones touchpad or vanilla Windows computer. So as of 2016 the next thing that I'll try to see, how my clients will react to it, is to say that the application will run only, when there is access to a proper server and unless they want to run that server at their own LAN, the application will run only, when there is internet access. If they do not want to run their own server with the VirtualBox appliance that is part of the project deliverables, then after the demo period ends they have to start paying for hosting. If they are too lazy to use the free, private, autonomous option, then they pay some reoccurring, nasty, hosting fees. I am totally honest by saying this nastyness to them at the very start and I will be very helpful at setting up those expensive hosting accounts. It's their choice. The application software deployment architecture is explained at the following image.

Please click the image
to see a larger version of it.
xxx Thank You for reading this text.