Saturday, September 18, 2021

Python and Javascript shared file on Google App Engine / Google Cloud Platform

The principle known as 'single source of truth' (SSoT) is quite important in computing, and is perhaps underappreciated. We need to write programs whose assumptions are consistent and easily discoverable. It's important to avoid the introduction of multiple sources for the same 'things': things that can be as simple as constants, and as complex as the definition of terminology within a theory of automation.

Sometimes, we need this to take place within a single shared file, accessible during runtime between the server and the client code, to express important editable values once.

I use local files in my development environment, to make changes to my webapp. In one important application, I don't need a makefile-like compiling stage. So, I test locally with a development server and a browser, as everyone does. But I want to have configuration files -- central, essential, important files -- which pretty much define everything critical in my application, both the universal and the contingent aspects ... and all the intermediate levels of detail needed to properly define those aspects of the work.

I call this an 'essence' or 'central description'. It is neither javascript (well, it's JSON here for technical convenience) nor python. It's just a configuration file. I've found that this compels me to maintain a quite useful habit: keep things modularized and parameterized at a higher level than code, making important ideas resuable as the application unfolds.

But how do you connect a Google App Engine Python server application (on Google Cloud Platform) to the javascript that it serves up? It seems like one should be able to use a shared JSON file easily. But that's not the case.

However, given that it's extremely important, here are the technical details of connecting them.

The shared file is shared.json.
The server-side python is called my_app.py.
The client-side javascript is my_app.js.
There's an app.yaml.
They are all in the same folder.
And I'm intentionally ignoring the html necessary to get to the javascript code, which could be embedded script, or a separate file, static, or generated ... that's all up to you. I don't know where you run your javascript, so I've just called it 'my_app.js'.
shared.json

{
 // a comment
 "name":"my_app",
 "things": {
 "one":"a"
 "two":"b"
 }
}


my_app.py

import webapp2
import io
import json

# this version of strip_comments 
# only works if the comment is defined
# by //, and // is the first non-whitespace
# on the line

def strip_comments(some_json):
 some_json = str(some_json)
 return re.sub(r'(?m)^[ \t]*//.*\n?', '', some_json)
    
class ReadAndRespond(webapp2.RequestHandler):
 def get(self):

 # get the file
 file = io.open("shared.json", "rb", buffering = 5)
 json_string = file.read()
 file.close()
 
 # strip the comments
 json_sans_comments = strip_comments(json_string)
 
 # load the JSON into a python dictionary
 dict_obj = json.loads(json_sans_comments)
 
 # get one of the values 
 my_things = dict_obj.get("things")
 
 # use it for a new JSON payload for the response
 # to the javascript request
 return_string = json.dumps(my_things, sort_keys=True, indent=4)

 self.response.write(return_string)

app = webapp2.WSGIApplication([('the_json', ReadAndRespond)])


my_app.js

 ...

 var fetch = eval( ajaxReturn('the_json') );
 var things = fetch;

 ...
 
 function ajaxReturn(xhr_url) {
    var return_string = '';
    $.ajax({
                dataType: "text",
                url : xhr_url,
                async: false,
                success : function (newContent) {
                   return_string = newContent;
                },
                error : function ()
                {
                }
    });
    return return_string;
 }
 
 ...

app.yaml

runtime: python27
api_version: 1
threadsafe: false

handlers:
- url: /shared.json
  static_files: shared.json
  application_readable: true
  secure: always
  redirect_http_response_code: 301
  
  - url: /.*
  script: my_app.app
  secure: always
  redirect_http_response_code: 301
  

Tuesday, March 23, 2021

Google Cloud Platform or Google App Engine 'gcloud app deploy' not updating your app? Debugger not working?

If you use python, and occasionally run your app locally, you'll notice some files that are generated: .pyc and index.yaml. If you accidentally 'gcloud app deploy' these files, especially when you have not run it locally for a while, they can break your application! Or drive you crazy!

In particular, they break the Google Cloud Console debugger (formerly the stackdriver debugger)  ... so while you're trying to figure out why your application doesn't work as expected, the debugger keeps telling you that you're setting logpoints on non-existent lines of code!

What has happened, is that Google is not checking to see whether the .py source file and the .pyc file match. It would be very nice if they did this. They could just check the timestamps. It would be easy. But they don't. 

And you keep changing your source code in trivial ways, perhaps adding some logging to see what's going on, but the live deployed application doesn't change. And the debugger doesn't work!

The fix is obvious: delete the .pyc and index.yaml files. Or put them in your .gcloudignore file.

I hope this helps someone. :-)




CHAQ: Central Handler, Action Queue

The evolution of cheq into CHAQ is pretty easy to explain. 'Events' are things that happen to you, and 'actions' are things that you do. The Central Handler in a javascript application is called by system events that we initiate in our own code, and then we let go, so the browser can do what it needs to. But before we let go, we check what actions we need to take. They are on a queue of these actions, which we've loaded within our own code. We process them, then pass control to the browser.

You can see this code, used to explain itself in a baby web app, at chaq.rocks.  

One other change -- our 'option-oriented programming', where the program itself is determined by a JSON structure, refers to the names of other JSON objects, not to functions. I found that I would otherwise not make the functions generally usable by other JSON instructions. So functions are only called by 'process'. Ultimately, for namespace sanity, our functions are injected into process, where they are called by the program described in JSON, which I call the 'essence', and which is really a not-javascript-specific declarative programming language 'structure', which can be used for any level of abstraction you like.

The point of the essence is to give people an opportunity to maintain clarity of ideas in the description of the program's activity. Coding beyond the barrier of incomprehensibility (and other desired qualities) is very common in software development. Anything that helps us to prevent that, needs to be explored further.