Monday, August 01, 2016

cheq: central_handler and event_queue

No matter how you look at it, even if it's hidden, every good browser-based JavaScript application must have a "central handler", and an "event queue". I call this necessity "cheq", as a mnemonic.

Why is this? Because a JavaScript program cannot hold onto (monopolize or block) the execution thread (the control flow of the browser's computational actions) and still make use of the essential services provided by the browser: rendering, user event handling, etc. We must pass control back to the browser, all the time, or nothing apparently happens.

But how do you do this, if you need your program to do "many things that are tied together", while passing control to the browser between each of these things? The answer, as I've said, is your own "event queue": a control channel under your control, which will persist while the browser is busy, say, rendering something for you. Every JavaScript programmer runs into this problem all the time: why isn't "X" appearing on the screen? Oh -- I didn't pass control back to the browser. This is especially obvious when you build animations.

If you have an event queue of your own, independent of the browser's event system, then you need a central_handler that manages that event_queue. Hence "cheq":


/* -----------------
 cheq.rocks 
 "cheq" means "central_handler event_queue".

 This needs to be at the heart of any browser-based 
 javascript application. It allows you to control your 
 program flow while cooperatively passing control to the 
 browser in order to render, handle events ... 

 The initial call from index.html looks like this:
     event_queue.push({'name':'initial_subhandler_name',
      'delay':2000});
     central_handler()

 Subsequent calls from inside the event look like:
     event_queue.push({'name':'some_subhandler_name',
      'delay':2000});
     central_handler()

 OUR EVENT QUEUE (so the browser regularly gets control):
  uses .push(x) to add to queue
  and .shift() to get the next event
*/
var event_queue = [];
var the_event = null;

// CENTRAL_HANDLER:
//  called by onload and setTimeout
function central_handler() {

    if (!(event_queue.length > 0)) {
 return;
    }
    the_event = event_queue.shift();

    // call event
    window[the_event.name]();

    // only loop until the stack is empty
    if (event_queue.length > 0) {
       setTimeout(function () {central_handler();},
    the_event.delay);
    }
}

// end of cheq.rocks
// -----------------


"Cheq" may be considered "the heart" of any JavaScript application, from one perspective. It's not necessarily the most useful idea for a program "heart", for comprehensibility. But ... maybe it is a useful central organizing principle. You never know until you try. So, I'm going to try. I'll evaluate this with my "smoothly unfolding sequence" approach, described at core memory, making good use of the Life Perception Faculty in the human brain as a means of judgment, and see if I can maintain "explanatory" reasonableness as well. My explorations in maintaining a good development structure, from this starting point, will be here: cheq.rocks

Wednesday, June 22, 2016

The Biology of Password Security

If you can, try to accurately recall a complete sentence that you just said, to another human being. One that wasn't very long, nor very short, nor a cliché, nor a quote. One that was grammatically correct to you. 

That sentence is more than likely unique in human history. At the very least, if you type it into Google, with quotes around it, you are unlikely to find it. Try it a few times. 

People are often under the misimpression that all human sentences must be on the internet, making up a kind of corpus of all languages. Nothing could be further from the truth. Any natural language 'corpus' is a finite set of captured sentences: they are superficial artifacts of the complex human thought that produced the sentence. Even if a corpus was, somehow, an infinite set of sentences, it wouldn't be the right infinite set, because we don't yet know what that set is.

It's not possible to get a machine to automatically generate your phrase. There is no generator for all (and only) the sentences of any human natural language, for the following very simple reason. The mechanism that produces language is in the human brain; the brain is an extremely complex biological system, and we understand some things about it, but not much. It is highly structured, but we do not know the structure, and in fact we only have a few dozen reliable hints about the structure, despite centuries of intense work by legions of linguists. Since that biological structure is a major factor for any natural language grammar, we have no worked-out grammar (syntax), in the sense of an explicit definition of an infinite set of sentences for any human language. The actual grammar is a faculty of our brain, the faculty we use to both generate sentences and evaluate whether something is grammatical. It is part of our biology, and we have no more conscious access to its detailed operation than we do about our visual system, or, for that matter, our digestive system. We must construct experiments -- testing this biological grammatical 'meter', this language faculty, in the same way we construct experiments on our visual system with optical illusions -- in order to find out things about its operation. 

This is a research initiative, and we'll all be dead before the human natural language mechanism is understood well enough to create a generator for all and only natural language sentences. 

So, it's quite safe to take any natural sentence, like this one [but not this one, of course, since it's been written down!] and use it as your password. (These are also known as passphrases).

It's also easier to remember natural sentences than all this nonsense. But it's not trivial to remember natural sentences. You need to train yourself, and learn to be sensitive to your own speaking, including speaking to yourself (we mostly use our language faculty to talk to ourselves). If you're a writer or actor you might already have practiced this facility for remembering sentences, which we often call an "ear". But anyone can do it. It's part of our genetic endowment.

If we could develop a culture of language sensitivity, we'd have far fewer problems with passwords. Those silly and unnecessary "trick" password-generators would then become a thing of the past.

One more note about infinite sets, because there's a misconception about them. There are an infinite number of infinite sets, but a very limited number of infinite sets that are human languages ... at most a finite multiple of all the people who ever lived. 

A hypothetical infinite set can be aspirational (all the sentences of English), but an actual infinite set requires a generator function. We can prove that there's an infinite subset of sentences within English ("this, and that, and this ..."), proving that the hypothetical full set is infinite. But we don't have a generator for all and only the sentences of English, or any of the other billions of languages that ever existed (assuming, again, that the upper bound is some multiple of every individual, with somewhat unique human languages of their own).

However, the joy of natural science is to discover more about the structure that is universal in all this variation ... "universal grammar" just means that part of language that is our genetic endowment. In this sense, every human language is the same. And until we understand the universal grammar, we cannot have a single complete generator for any particular natural language. Note also that we use our brains to generate what to say: language is, after all, the expression of thought. So until we understand thought, we won't be able to generate all-and-only sentences of any language. 

So, again, your real human sentence is safe from hackers. (That would have been a good one.) 

Sunday, May 22, 2016

Adding a docker service to boot on Linux

If you want a docker container, or a service that starts docker containers, to startup on a system boot, the correct method is buried in the manual for the command systemctl.

The system boot technology on GNU/Linux is shifting. Many distros are no longer relying on the Unix System V approach, with the oddly named /etc/init.d directory and the rcX-type 'run control' directories for different run levels. 

The move is instead towards systemd and upstart. I'm only talking about systemd here, and the best way to install a service that depends upon docker

Systemd requires unit files installed, in the proper way, in its /etc/systemd/system directories. Here's an example of a unit file, which starts up the mediawiki containers service mentioned in the previous post:

[Unit]
Description=mediawiki-containers
After=docker.service
Requires=docker.service

[Service]
TimeoutStartSec=0
WorkingDirectory=/srv/mediawiki-containers
ExecStart=/srv/mediawiki-containers/mediawiki-containers start
ExecStop=/srv/mediawiki-containers/mediawiki-containers stop

[Install]
WantedBy=multi-user.target

This file is named mediawiki-containers.service, but it shouldn't be confused with any script service files, including the one referred to (mediawiki-containers). This unit file is called a service file because it refers to other kinds of services.

Clearly it must be started after docker is up.


Now, there's a directive in here to install it to the 'multi-user target'. This is a kind of run-level definition. But it adds to the trickiness of actually installing this.

I'm installing it on Ubuntu Linux 15.10, which has systemd pre-installed.

This unit file was authored by the mediawiki folks, and they put it on your system here: /srv/mediawiki-containers/init/mediawiki-containers.service ...


So all that's needed is to tell systemd about it, using systemctl enable:

# systemctl enable /srv/mediawiki-containers/init/mediawiki-containers.service

Created symlink from
/etc/systemd/system/multi-user.target.wants/mediawiki-containers.service to
/srv/mediawiki-containers/init/mediawiki-containers.service.

Created symlink from
/etc/systemd/system/mediawiki-containers.service to
/srv/mediawiki-containers/init/mediawiki-containers.service.

#

It should then startup on the next reboot.

Tuesday, March 15, 2016

Modifying mediawiki-docker: adding LDAP to the mediawiki container

The 2008 release of the 2.6.24 Linux kernel opened a new world. In particular, it supported a lightweight option for encapsulating machine configuration. 

The new kernel features -- 'namespaces' and 'control groups' -- can meet our human need for a virtual machine, without the overhead of simulated machines or additional copies of an operating system.

A 'container' is a single process, which is apparently running on its own hardware. We used to do this with a 'jail' or 'sandbox', which you could access with chroot, improved by later tools and kernel patches. The term container arose with the new, better, standardized isolation, and software that easily moves these environments around.

The premier open-source management system for containers is Docker:

If you manage or build applications and services on Linux, and you want to save your work, you need Docker. Once you see what it does, you'll want it.

Let's look at using Docker for one application. 

MediaWiki is produced by WikiMedia. You use MediaWiki everyday, because it's the free software behaind wikipedia.org.

The good people at Wikimedia created an experimental install script for a complete, running mediawiki docker deployment. It installs four interacting docker containers onto one machine -- with one line. 

On a Ubuntu 15.10 server, you can type:


curl https://raw.githubusercontent.com/wikimedia/mediawiki-containers/master/mediawiki-containers | sudo bash


This is a bash script, with both install and stop/start services, within the github project wikimedia/mediawiki-containers. If you look at the file, which is called mediawiki-containers, you'll see it pulls four docker projects from the docker hub:

  • tonistiigi/dnsdock
  • mariadb
  • wikimedia/mediawiki
  • wikimedia/mediawiki-node-services

And then it builds 'images'. What are they?

An image is a docker project that can be built, so it can then be run, as a container

The centerpiece of a docker project is the Dockerfile. It directs the construction of the image. It downloads resources to build the image. The images can be run as containers extremely quickly, because they are already built. If you try to 'docker run' a project that has not been built, docker will download the project and build it into an image for you.

If you want to modify a docker project, you might want to pull your changes forward, to the Dockerfile, as much as possible, or, in this case, perhaps to the mediawiki-containers service file (which gets installed in /srv/mediawiki-containers on your machine). Saving them in an image tends to get lost. It's nice to destroy containers when you stop them, and then run from a fresh image.

As an example: I needed LDAP in my docker mediawiki container. One way to do this is to modify PHP on the running image (adding php5-ldap).

# docker exec -it mediawiki bash
(container id)# (make your changes)
(container id)# exit

# docker commit -m "with my modification"

But that leaves you with unnecessarily large images to keep track of. 

Why not pull the change forward to the Dockerfile? mediawiki-containers 'pulls' the docker project for mediawiki, whose github source can be found here:

https://github.com/wikimedia/mediawiki-docker

Download it (with git, or in a zip, whatever) and add to the docker file.


# cd
# wget https://github.com/wikimedia/mediawiki-docker/archive/master.zip
# apt-get update
# apt-get install unzip
# unzip master
# mv mediawiki-docker-master mediawiki
# cd mediawiki
# emacs Dockerfile
Added php5-ldap to the appropriate spot
# docker build -t greg/mediawiki .
# docker images
# cd /srv/mediawiki-containers/
# emacs mediawiki-containers
Changed run to greg/mediawiki
# service mediawiki-containers restart
# journalctl -f -u mediawiki-containers -n 0 | sed '/: Done in /q'

If you add phpinfo(); to your /srv/mediawiki-containers/data/mediawiki/LocalSettings.php file, for a moment, you'll notice that LDAP is enabled. 

You now just need to configure the LocalSettings.php file to require the mediawiki LDAP extension and set wg values for your network.