Bootstrapps Table of Contents
- Intro to Programming
- Setting up a Programming Environment
- Language Concept Subutorials
- Physical Computers
- Types and Typing
- Control Structures
- Regular Expressions
- Programming Tools Subutorials
- File System
- Web Services
- Connecting Programs
- Understanding (the Domain of) the Problem
- User Interface
- Developer Interface
For a while now, I’ve wanted to write the sort of introduction to programming that would have really made a difference to me when I was learning programming. Actually, I understand that in writing it now, it will have perhaps an even more powerful impact. I have a natural talent (even if my other writing doesn’t always exemplify it to you) for appraising a context of information with respect to its clarity and completeness. Small, perhaps, in the overall scheme of things, nevertheless we must work with what we’ve got. The importance of my little talent will become clear as the tutorials distill into an essence most efficacious. You will see, I am going to bootstrap our way to the perfect programming tutorial by constantly reappraising and refactoring the product towards ever greater value. And in this way, the subtler skills of the master programmer shall be introduced and encouraged.
There’s a difference, however, between this “tutorial” and others. It is one of scope, of vision, of intent. Most tutorials are just that: “tutorials”. Their author has set the scope of their project to be a collection of documents describing and showing how to do or make some activity or thing. Perhaps there are ancillary materials and tools. Perhaps even a kind of interactivity. My question is, how far can the capabilities of programming be taken in a tutorial about programming? To what degree can I bootstrap (myself, you, whoever) into a master programmer?
It comes down to two things. The writer and the reader. The writer must be dedicated to putting the most into the presentation as well as continuous refinement, just as the reader must be dedicated to getting the most out of it and as well as continuous refinement. For the reader, this should be easy. My avowed goal is to catalyze your transformation into a master programmer. Not just a competent one. A master. Someone in the top half a percentile. Perhaps it’s a little more difficult for the writer, however. Is this some sort of altruistic gesture? Am I toiling away solely for you benefit? No. For better or worse, I’m thinking globally, but acting locally. I’m going to transform myself into a master programmer (I am not one, right now, I assure you, which isn’t to say that I’m no good, but rather that I have a pretty lofty perception of what constitutes a master programmer). I am going to record my doing of it. I am going to feed the process back in upon itself. I am going to take a picture of a lightning bolt as it arcs through the sky.
You see, I think programming is a multifaceted activity. Programming, like any craft, is not just the end product. Which is to say, a full description of how to get from a blank screen to a working program does not simply involve a description of the meaning of the keywords and syntax of the programming language used. There is much more involved in arriving at a “program”. And if you really want to master arriving at “programs”, well, you’ve got your work cut out for you. You my as well take up Yoga too, and settle in for the long haul.
Programming tends to involve (but certainly isn’t limited to):
- Excitement about creating real, active solutions to real, active problems
- Understanding “int x = 1”, otherwise known as “variable assignment“;
- Understanding the keywords, operators and syntax of a programming language
- Understanding Control Flow statements (if, foreach, while, et cetera)
- Code formatting
- Being able to analyze problems in terms of smaller problems
- Being able to stitch together the solutions to many small problems into a big solution to the overall problem
- Being able to see generic patterns in specific instances
- Design Patterns (Gang of Four and beyond) and
- Being able to constrain a solution to use generic patterns
- Understanding and applying user interface best practices
- Refactoring code to arrive at better solutions and design patterns
- Ability to understand knowledge domains (Domain Driven Design and a pdf of Domain Driven Design Quickly)
- Being able to create tests (test driven development)
- Object Orientation, Analysis, and Design
- Source Control (Git, CVS, SVN…)
- Object Relational Mappers (ORMs)
- Integrated Development Environments (IDEs)
- Command Lines
- UML (Unified Modeling Language)
- Security and Permissions
- Generics, Templates
- Best Practices
- Static vs Dynamic and Interpretted vs Compiled programming languages
- Functional vs Procedural vs Object Oriented programming languages
- Computer Hardware
- Web Services
- Regular Expressions
- Databases (SQL, NoSQL, etc)
- XML, JSON, CSV (data exchange formats)
- Communication with other programmers
- Communication with non-programmers
- Clear thought and writing
- Being able to search for solutions to any sort of problem encountered while composing a program
- Ability to step away from problems
- Domain Specific Languages
- Continuous Learning
- Clean Code
- Legacy Code
That’s already a lot. Traditionally speaking, it’s five years of knowledge, to attain a passing acquaintance. But you know what. That’s also totally false. That’s five years of textbooks. Five years of classrooms. Five years of armchair reading in front of the TV.
It’s a year of engaged learning. Let’s begin.
A few words on the project in general before going into programming specifically. I am going to try to quickly (crudely) put together the structural scaffolding of this project (all the basic tutorials) and then go about refining them. In general, I would expect a year to complete the project and 3 months for it to be generally usable. If any post in the series begins with [Provisional] than take everything with a grain of salt, I don’t consider it “released” yet. Alternatively, I will begin each vetted tutorial in the series with a version number in brackets: [1.0] will be release one of a tutorial. The tutorial as a whole will consist of variously versioned subutorials. “Metaprogramming” may be at version [4.112] while “Physical Computers” languishes at [1.0] (reflecting in general my interests at the moment).
Let’s set our anchor in the wind.
What is programming? There are many ways to answer this question. The right answer depends on the listener. Programming is a very mercurial referent One could even doubt a fundamental difference to exist between the programmer of “software technology” and the engineer of “material technology”. Or replace “programmer” with “hacker” or “engineer” with “artist” or “tinkerer” or “architect”. It all comes down to a fundamental human activity, structured control. Now, “control” is an overloaded word, but there is a quintessence to its various senses. Creating precise or imprecise workflows of will. This isn’t the place to start defining “will”, but we all have some sense of the fact that it is a fact that we can form an idea of some action and/or do some action. “Structured ‘doing'” comes close. Plenty of other places in this blog deal with the nature of “being” and “doing” and “structure” and “function” and “collections of doing” and “organisms” and “spells” and even “programs”.
Programming involves spelling out interconnected, ordered instructions.
A relative(ly accurate) distinction between architecture and programming could be made in relation to the fact that buildings are static solutions to environmental problems, whereas programs are self-modifiable solutions to problems (of course, buildings could become self-modifiable as well, but probably because programs wound their way into the material and connections of the building).
Programs are, essentially, interconnected algorithms that can interconnect. In The Algorithm Design Manual we learn right off the bat that:
What is an algorithm? An algorithm is a procedure to accomplish a specific task. An algorithm is the idea behind any reasonable computer program.
This begins to hint at a second face of programming. Not only do we have the program, but we have the “specified task”. Specifying a task involves analyzing its parts. Programming is the building up of a system of algorithms. In other words, a program is a structured associations of tiny solutions to tiny problems. Programming is building up. But it is something that is built up around something else. A scaffolding. Do you see it in there? Those “tiny problems”. As much as programming is a building up (of a codebase, et cetera), it is also a breaking down (of a problem into parts). It is important to keep this in mind. Too often programmers get stuck on the product of programming, ignoring the means.
The object of philosophy is the logical clarification of thoughts.
Philosophy is not a theory but an activity.
A philosophical work consists essentially of elucidations.
The result of philosophy is not a number of “philosophical propositions”, but to make propositions clear.
Philosophy should make clear and delimit sharply the thoughts which otherwise are, as it were, opaque and blurred.
Now I’ll move on to Harold Abelson and Gerald Sussman in SICP (Structure and Interpretation of Computer Programs), none other than the defacto introduction to programming computers at MIT for decades (although not necessarily anymore [also check this video series out from HP back in the day! It’s super awesome]):
We are about to study the idea of a computational process. Computational processes are abstract beings that inhabit computers. As they evolve, processes manipulate other abstract things called data. The evolution of a process is directed by a pattern of rules called a program. People create programs to direct processes. In effect, we conjure the spirits of the computer with our spells.
A computational process is indeed much like a sorcerer’s idea of a spirit. It cannot be seen or touched. It is not composed of matter at all. However, it is very real. It can perform intellectual work. It can answer questions. It can affect the world by disbursing money at a bank or by controlling a robot arm in a factory. The programs we use to conjure processes are like a sorcerer’s spells. They are carefully composed from symbolic expressions in arcane and esoteric programming languages that prescribe the tasks we want our processes to perform. (pg 1)
I’m not making this stuff up.
Let’s examine what programs can “look like”. Well, they can look like different things, depending on what instruments you’re using to look. A program can look like a collection of files in a (hopefully) intent-revealing directory structure:
Or it may look like traditional code:
Or maybe some Class Diagrams:
Or maybe as a work flow:
To me, a program is like a piece of text with the purpose of accurately engendering a specific meaning the mind of the reader. It basically consists of well-named and well-defined definitions. Programs are highly interconnected in that a typical programming involves naming a tiny process and referring to it within the definition of other tiny processes. In this way we can build up layers of inter-supporting abstractions. In the same way that a single word can carry with it whole contexts of meaning that, without the word, would take paragraphs to describe, so too can programs be built out of nested definitions.
Imagine reading a document where you could click on any word and navigate to its definition. That definition would be composed of words as well and any of those could be clicked on to navigate to their definition. And so on. In the end, this is very much like what a program is. Every word used in a program (even human text, like “Jack and Jill went up the hill…”) ultimately has a definition in terms of other words that the computer (program compiler) has a definition of, all the way down to the tiniest processes of all, individual changes within the electronic circuits themselves (FYI, that is the difference, for instance, between X86 processors and ARM processors)
So, how do changes to collections of present/absent electrical pulses turn into letters and text and pictures and user interfaces and meaningful content? Trust me, these days programmers don’t instruct the computer about the color of each and every pixel on the monitor and program at the level of describing the color and intensity of pixels on the screen. These days programmers communicate in terms of preexisting abstractions that make the solution to the problem of, for instance, creating an interface into the details of a purchase order, much easier and closer to the level that people actually think about the purchase order, rather than the nitty gritty level electrical hardware signals.
An example is due. Let’s say we want to tackle the project of creating a text editor (not for the faint of heart). At a basic level, this involves showing and saving a series of keystrokes by the user. When a writer creates a document, essentially they are linearly stringing together characters (symbols, letters, shapes) that they can move around in and edit over time. The characters are specified by pressing keys on a keyboard, either singly or in combination and the navigation happens through keyboard and mouse and touch and whatever essentially positions a “cursor” where the next character is to be inserted in the string of characters. While that is accurate in certain reductionist ways, if we want our text editor to be useful to the modern writer we are going to have to add functionality that operates in terms of higher level concepts than “characters”. For instance, spell check involves “words” and formatting may involve “sentences” and “paragraphs” and “pages” and “margins” and “fonts” and “footnotes” and “lists” and “text selections” all the sorts of stuff that writers use to structure and interact with various aggregations of characters in a text document. Once you’ve defined these terms in your program, you can write functionality that uses them and exists in relation to them.
I want to emphasize the importance of organization to a program. In the end, any program that works (however that is defined for that program) is organized. It’ can’t help but be due to the way processors work. The only question is how well that organization is understood by they who wrote it and how well it can come to be understood by those who would understand it.