Pogoscript

How do I?

Ok, so you want to know what this pogoscript thing is all about… It’s a programming langauge, you probably knew that already. It compiles to JavaScript too, you probably also knew that. Because it compiles to JavaScript it’s like JavaScript. Most, if not all of the language’s behaviour is identical to JavaScript, pogoscript is merely a new syntax for JavaScript, plus a few interesting things which we’ll get to.

Variables

Variables can be declared like this:

windSpeed = 25

Once declared, you can then reassign them using the := operator:

windSpeed := 13

There is another interesting thing about variables in Pogoscript, and that is that they can contains spaces. The above variable can also be written like this:

wind speed = 25

This, as we’ll see next, is useful when defining and calling functions.

Functions

Variables are the building blocks, but functions have all the fun. In Pogoscript functions are called by passing arguments. Arguments can be numbers, strings or other expressions like variables or indeed the results of other functions.

is (windSpeed) strongEnoughForMyKite

Here we called a function named isStrongEnoughForMyKite, but we passed the windSpeed argument in the middle of the function, between the is and the strong. This is possible because we can also write the function like this:

is (windSpeed) strong enough for my kite

Identifiers in Pogoscript can be written in both space-case and camel-case. The actual position of the arguments does not matter, just the order. In fact, we could call the same function like this:

(windSpeed) isStrongEnoughForMyKite

Or

isStrongEnoughForMy (windSpeed) kite

Neither of which read as well as the first, but are nevertheless effectively the same thing.

Eventually, you’ll want to define your own functions too. Like variables, we use equals =, like this:

is (windSpeed) tooStrongForMyKite = windSpeed > 30

We can also drop the function’s body onto an indented line like this:

is (windSpeed) tooStrongForMyKite =
    windSpeed > 30

The last statement in a function is the functions ultimate value.

These are simple functions, but there are other ways functions can be called, and it’s all to do with how the arguments are written. For example, lets imagine we have a function called sum, it returns the sum of the numbers you pass to it.

sum 1 2 3
=> 6

Notice how the numbers didn’t need parenthesis? Numbers and strings don’t need to be in parenthesis, but there’s nothing stopping you from using them:

sum (1) (2) (3)
=> 6

In fact, you can use brackets for some arguments and not for others:

sum (1) 2 (3)
=> 6

If two arguments are next to each other, they can be in the same parantheses and separated by commas:

sum (1, 2) 3
=> 6

Or

sum (1, 2, 3)
=> 6

Now all of the above is true if there is a name, in this case sum. It’s time to introduce some new terminology to help us: all of the examples you’ve seen so far are forms. A form is a list of words, numbers, strings and expressions in parentheses. (They can also contain blocks and parameters, but we’ll get to that soon.) A form has a name if it contains some words, like our sum example above. If a form doesn’t have a name, for example, if it’s all arguments like this:

(sum, 1, 2, 3)

Then the first argument is taken as the function and the remaining arguments the arguments to that function. So the following forms are functionally identical:

(sum, 1, 2, 3)
sum 1 2 3
(sum) 1 2 3

If there’s only one argument and no name, then that argument is taken as a value, not a function, and it’s not called. So the following are identical:

(windSpeed)
windSpeed

Blocks

Blocks are arguments to functions that are also functions themselves. In effect, blocks allow you to pass some custom behaviour to a function, for example, to run a bit of code if an event occurs or to answer some question required during the function’s execution. This is a classic functional programming design pattern, and a function that takes an other function as an argument is often called a higher-order function.

In pogoscript, blocks come in two syntactical forms, the indented version looks like this:

loop 2 times
  console.log 'hi' 

This, as you can imagine, prints hi twice.

The other form looks like this:

loop 2 times @{ console.log 'hi' }

In fact, anywhere you see an indented section of code, you can rewrite it using the @{ ... } form, and visa-versa.

Higher-Order Functions

Don’t worry, higher-order functions aren’t special. All they do is call one or more of their arguments as functions. So, for example, to define the loop times function we used above, we could write this:

loop (times) times (block) =
  for (n = 1, n <= times, ++n)
    block ()

It just calls block inside the for loop.

Blocks that have parameters

Blocks can also define parameters, that is, as functions they can be called with arguments and so they can receive those arguments too. To define block parameters we use the @(param) syntax. Say, we want to print the number of the loop in our loop times function, starting with 1.

loop 2 times @(n)
  console.log "#(n): hi"

Of course, we’d have to change our definition of loop times to pass the loop number to block:

loop (times) times (block) =
  for (n = 1, n <= times, ++n)
    block (n)

Multiple parameters are also allowed and they can be separate, as in @(a) @(b), or together separated by commas, as in @(a, b). So, lets imagine we have a list of items that we want to loop through, one for each item, but we want the index too. First let’s define for each item and index in:

forEachItemAndIndexIn (list, block) =
  for (index = 0, index < list.length, ++index)
    block (list.(index), index)

Then:

primes = [1, 2, 3, 5, 7, 11, 13]
forEachItem @(prime) andIndex @(index) in (primes)
  console.log "Prime number #(index + 1) is #(prime)"

Passing multiple blocks

Several blocks can be passed to a function. This doesn’t often happen but can be used to describe some complex behaviour. Lets imagine that you wanted to loop n times, but if it didn’t loop (e.g. if n was zero) then you wanted to do something else.

loop (n) times
  console.log 'hi'
otherwise
  console.log 'is anyone here?'

This is equivalent to and could also be written like this:

loop (n) times @{ console.log 'hi' } otherwise @{ console.log 'is anyone here?' }

Or any combination thereof. But there’s a trick here, if you write it like this, with an empty line before the otherwise:

loop (n) times
  console.log 'hi'

otherwise
  console.log 'is anyone here?'

Then it is interpreted as being two separate calls, one to loop times and the other to a (perhaps non-existent) function called otherwise. People bump into this when they first start writing pogoscript but they quickly get over it. The thing to remember is that the line immediately after a block is taken as the continuation of the line that started the block.

Chaining calls with blocks

Now that we know about the line immediately after a block, we can use that to chain calls with blocks. The common scenario is a map/filter, where we map some values from one list into another, then apply a filter:

mapEach @(contact) in (address book) to
  contact.firstName
.selectEach @(first name) where
  firstName.match r/^a/i

Which is the same as this:

mapEach @(contact) in (addressBook) to @{ contact.firstName }.selectEach @(firstName) where
  firstName.match r/^a/i

As a side note, the above can be written using list comprehensions like this:

[contact <- addressBook, firstName = contact.firstName, firstName.match r/^a/i, firstName]

Or this might be better:

[
  contact <- addressBook
  firstName = contact.firstName
  firstName.match r/^a/i
  firstName
]

After the block

In general, the line after the block is part of the same function call, so you can pass additional arguments:

setTimeout
  console.log "a belated greeting"
1000

Or, as a variable:

n = 1000

setTimeout
  console.log "a belated greeting"
(n)

Or you can continue describing the name of the function:

set
  console.log "a belated greeting"
timeout 1000

Or you can start with the block:

@{
  console.log "a belated greeting"
} setTimeout 1000

(both awkward ways to call setTimeout but still valid.)

Optional Arguments

Some function calls can get away with just having good defaults, and not require everything to be configured ad tedium. For these sorts of functions, optional arguments can be used.

Say we have a function to transcode CD quality audio into bandwidth friendly MP3:

convert (audioFile) intoMp3

This function would come with some useful defaults for bitrate, say 128kbps. But no doubt at some point we’d like to be able to override that, so we could call it like this:

convert (audioFile) intoMp3 (bitrate = 320kbps)

This passes an additional named argument to the function. We write 320kbps, which is a call to a function called kbps:

(n)kbps = n * 1024

The optional argument is passed in an object as the last argument, so you could write it like this:

convert (audioFile) intoMp3 { bitrate = 320kbps }

In fact named arguments are always passed as the last argument, so this:

(bitrate = 320kbps) convert (audioFile) intoMp3

Is identical to the original call.

This helps if we’re calling a function with a block:

time
  someLongOperation()

Or

time (message = 'the long operation')
  someLongOperation()

The order of the arguments passed is identical, with the difference being an additional trailing options argument.

We’ll cover objects in more detail in the next session.

To accept this argument, for example, if we were defining this function, we’d write something like this:

convert (audioFile) intoMp3 (bitrate = 128kbps) =
    // code to convert an audio file into mp3 with bitrate

When the function is defined, the default value can be specified, as the value of the function call 128kbps is above.

Splat Arguments

Some functions can accept an arbitrary number of arguments. Our hypothetical sum above was one of them. To acheive this, we use splats, which is some Ruby terminology for being able to access a number of arguments as one.

Lets finally implement our sum function:

sum (numbers, ...) =
    sumSoFar = 0

    for each @(number) in (numbers)
        sumSoFar := sumSoFar + number

    sumSoFar

When we put some ellipses after numbers, we’re saying that numbers represents all the arguments passed to sum. Likewise, if we had a list of numbers that we wanted to pass to sum when we called it, we could also use the ellipses:

ourNumbers = [1, 2, 3]
sum (ourNumbers, ...)

Which would be the same as:

sum 1 2 3

When defining a function, we can only define one splat parameter, but we can continue to have other non-splat parameters before and after it. For example, the following would work:

sum (first, second, middleNumbers, ..., last) =
    // ...

But for calling a function, we can specify as many splats as we’re happy to:

smallNumbers = [1, 2, 3]
largeNumbers = [7, 8, 9]

sum (smallNumbers) ... 4 5 6 (largeNumbers) ...

The only other restriction when defining functions is that splat parameters cannot be used alongside optional parameters - the following is not allowed:

sum (args, ..., quickly = true) =
    // ...

The reason is that since optional arguments are optionally passed, we can never be sure whether an argument belongs in the splat or if it’s the optional argument. A technical detail, but nonetheless restrictive.

Objects

Functions are of course objects in their own way: they can encapsulate state (by referring to variables in lexical scope); they offer an abstract interface (you can call them by passing arguments); they are polymorphic (you can’t really tell what implementation you’re calling, and nor should you); and they can inherit the behaviour of other functions in delegation, arguably an analogue to object oriented inheritance.

But sometimes functions are not enough and something more is needed. What objects offer is the ability to invoke different but related operations, whereas functions can only invoke a single operation.

Objects in Pogoscript are, naturally, no different to objects in JavaScript, they just use a slight variation on the syntax:

person = {
  name = "Jean Baudrillard"
  hobby = "Philosophy"
}

There’s also a shorthand, omitting the =:

person = {name "Jean Baudrillard", hobby "Philosophy"}

Methods follow the same syntax as functions:

person = {
  name = "Jean Baudrillard"
  hobby = "Philosophy"

  saySomething () =
    console.log "It is always the same: once you are liberated,
                 you are forced to ask who you are."
}

Of course, you can define fields and methods from outside the object too. We could write the above like this:

person = {}

person.name = "Jean Baudrillard"
person.hobby = "Philosophy"

person.saySomething() =
  console.log "It is always the same: once you are liberated,
               you are forced to ask who you are."

Accessing fields on objects is fairly predictable:

person.name

Calling methods on objects is also fairly predictable:

person.saySomething()

Field Expressions

You can also access a field using an expression in parenthesis:

fieldName = 'name'
person.(fieldName)

And similarly for methods:

methodName = 'saySomething'
person.(methodName)()

The same is true for defining fields and methods. For example, you can create a hash object by adding values and keys:

peopleByName = {}

addPerson(person) =
  peopleByName.(person.name) = person

personByName(name) =
  peopleByName.(name)

Javascript doesn’t have classes like other programming languages, but it does objects and in a beautifully simple form. For example, where you might reach out for a class in another language, in Pogoscript (and Javascript) you can just create a function that returns a new object:

personNamed (name) = {
  name = name
  sayHi() = console.log "Hi, I'm #(self.name)"
}

joe = personNamed "Joe"
joe.sayHi()

Prototypes

This is a very simple way to create objects, and every object created by this function will contain a sayHi method, technically not the same method, but identical in behaviour. Another way to share methods and values between objects is to use prototypes. When you create an object with a prototype it inherits the prototype’s properties and methods.

You can create a prototype with the prototype function:

person = prototype {
  sayHi() = console.log "Hi, I'm #(self.name)"
}

personNamed (name) = person {
  name = name
}

joe = personNamed "Joe"
joe.sayHi()

Here we defined a new prototype containing the sayHi method, then we created a new object with that prototype inheriting that method. The prototype person is a function that, when given an object will return a new object that shares all the properties of the person object.

Extending Prototypes

You can also make a new prototype that extends another, inheriting some properties and methods and overriding others.

person = prototype {
  sayHi() = console.log "Hi, I'm #(self.name)"
  age() = (self.dob - @new Date()).getTime() / 1000 / 60 / 60 / 365
}

australian = prototypeExtending (person) {
  sayHi() = console.log "G'day, I'm #(self.name)"
}

french = prototypeExtending (person) {
  sayHi() = console.log "Bonjour, je m'appel #(self.name)"
}

jack = australian {
  name = 'Jack'
  dob = @new Date(1995, 10, 3)
}
jack.sayHi()

laurant = french {
  name = 'Laurant'
  dob = @new Date(1993, 4, 24)
}
laurant.sayHi()

Which prototype?

You can ask an object if it inherits from a prototype using the :: operator:

jack :: australian
> true

jack :: french
> false

laurant :: french
> true

laurant :: person
> true

This is also true of other types too, functions, strings, arrays and objects.

Concurrency

Concurrency is one of the most fascinating aspects of computing. It offers so much, yet is so hard to do right. For a long time mainstream languages only offered the shared-state concurrency model, using mutexes to coordinate threads as they manipulated shared objects. It’s hard to overstate how notoriously difficult it is to make reliable concurrent software like this! It’s so hard to do that its very rarely done at all.

By contrast JavaScript is single-threaded, which certainly simplifies things, but how can you write concurrent applications in a single-threaded environment?

JavaScript uses an asynchronous programming model. This means that when you ask for an external resource, like the contents of a file or a database query, it sends a request for that resource and continues. It doesn’t block or wait for that resource to become ready. Instead you add a callback function to the request which is called when the resource is ready. When the callback function is called with the resource, it’s done on the same single thread, so you can manipulate the application state without needing mutexes or thread synchronisation.

Promises

Pogoscript models these asynchronous interactions using promises. Promises are a “promise to deliver a result in the near future.” When you ask for an external resource, you don’t receive the resource immediately, you receive a promise of that external resource.

promise = httpism.get "http://api.app.com/people/bob"

But a promise of a response on it’s own isn’t very much use to us, we need the actual response before we can do anything interesting…

The Resolve Operator: !

The ! operator is a very special operator in Pogoscript: it waits for a promise to turn into a real value. It does this by splitting your code into two bits: the code before the ! and the code after. The code after the ! doesn’t run until the promise is resolved. While it’s waiting your application can continue to handle other events. Let’s resolve that promise and do something with the response:

promise = httpism.get "http://api.app.com/people/bob"
response = promise!
console.log(response.body)

In terms of the asynchronous programming model:

  1. We start out by making a request to a HTTP server.
  2. In return we get a promise of the response, but nothing has happened yet.
  3. Then we use the ! operator, which, behind the scenes registers a callback function that will be called when the HTTP server responds.
  4. We wait for a few milliseconds. Our application can process other events in this time.
  5. The server responds, and the code after the ! is executed
  6. The promise resolves, returning the HTTP response
  7. We print out the body of the response

We can reduce our code into one line, resolving the promise inline:

bob = httpism.get "http://api.app.com/people/bob"!.body
console.log(bob)

Now if we want to wrap this up into a function, we can do that too:

printPerson(personId) =
  response = httpism.get "http://api.app.com/people/#(personId)"!
  console.log(response.body)

printPerson 'bob'

Now because the printPerson() function resolves a promise using the ! operator, it too returns a promise. We can use this promise to wait for Bob’s details to be printed:

printBobPromise = printPerson 'bob'
console.log "waiting for bob's details..."
printBobPromise!
console.log "bob's details printed"

More commonly we expect things to finish before proceeding so we use the ! on the function call itself.

printPerson 'bob'!
printPerson 'mary'!
console.log 'our favourite people...'

Here we print the details of both Bob and Mary, one after the other, then we print something nice about them.

To print out Bob and Mary’s details concurrently however, we can start printing each of their details, then wait for them both to complete:

bobPromise = printPerson 'bob'
maryPromise = printPerson 'mary'
bobPromise!
maryPromise!
console.log 'our favourite people...'

Avoiding Race Conditions

One of the problems with the above code is that we can’t be sure that Bob’s details will be printed before Mary’s. For all kinds of reasons, Mary’s details might be returned before Bob’s and hers will be printed first. It’s likely to happen 50% of the time in this case. This is a very common cause of bugs in concurrent software, and we call it a race condition when two or more operations are in a race to finish, and we can’t be sure which will win. Fortunately in our example the consequences aren’t terrible, but it’s wise to fix it anyway, if only to ensure that the user sees the same order each time they run our app.

What we’ll do is refactor our code a little. We’ll separate fetching the details from printing the details:

personDetailsFor (personId) =
  httpism.get "http://api.app.com/people/#(personId)"!.body

printPerson (person) =
  console.log (person)

Then we can get the details concurrently, but print them in order:

bobPromise = personDetailsFor 'bob'
maryPromise = personDetailsFor 'mary'

printPerson (bobPromise!)
printPerson (maryPromise!)
console.log 'our favourite people...'

Each of their promises resolve to their details, which we print. Doing it this way, Bob’s details will always be printed before Mary’s, no matter which one came back from the server first.

List Comprehensions

There is another way to do this too, using another Pogoscript feature called list comprehensions:

people = [
  personId <- ['bob', 'mary']
  personDetailsFor (personId)!
]

for each @(person) in (people)
  printPerson (person)

The list comprehension enumerates the people in the array ['bob', 'mary'] and calls personDetailsFor() on each of them. When we use the ! operator in a list comprehension it does this concurrently, not waiting for each call to complete before making the next one. However, it does wait for all of the people’s details to come back before returning them, and crucially, it always returns the people’s details in the same order in which they were enumerated, i.e. Bob before Mary. Then we enumerate the details and print them.

Promises and Functional Programming

As we’ve seen above, we can still write code that contains race conditions. The important thing to remember is that concurrent asynchronous calls can execute in any order.

Functional programming is a very useful tool in preventing race conditions. If our asynchronous functions manipulate application state, they can do it in any order and we are susceptible to race conditions. If, instead of writing functions that manipulate application state, we write functions that return new state, then we can ensure that we always get the same thing, regardless of the order in which events occurred. This is functional programming style, which is generally useful when writing software, but it’s also very useful when writing concurrent software.

For example, let’s say we want to download the details of both Bob and Mary so we can use them later:

peopleCache = {}

cacheDetailsOf (id) =
  details = personDetailsFor (id)!
  peopleCache.(id) = details

cacheDetailsOf 'bob'
cacheDetailsOf 'mary'

Here we declare an object peopleCache to hold the details, and cacheDetailsOf() to add the details of each person. The trouble with this code is that the peopleCache can be populated at any moment in the future. We can’t be sure when it will be populated, or even if it will be populated in case there are errors. Unfortunately this uncertainty is exposed to the rest of the application.

For example, if we had a button that sent a message to one of our users:

sendEmailTo (personId) =
  person = peopleCache.(personId)
  mailer.send! {
    subject = "hi"
    to = person.email
    body = "some details for you..."
  }

Unfortunately we now have a part of our application that assumes that peopleCache is populated. So if somebody clicked that button too soon, before the details were cached, it wouldn’t work.

A better way to do this is to use promises more explicitly. Let’s continue to use peopleCache as our cache, but let’s make it a promise of a cache. It might be populated, or it might not, but we can always ensure that it is populated before doing something with it. Instead of writing functions that manipulate our cache, let’s use a functional style and write a function that returns our cache:

detailsOfAllPeople () =
  bob = personDetailsFor 'bob'
  mary = personDetailsFor 'mary'
  {
    bob = bob!
    mary = mary!
  }

peopleCache = detailsOfAllPeople()

Then, when we need to retrieve somebody from the cache, we ensure that the cache promise is fulfilled by using peopleCache!:

sendEmailTo (personId) =
  person = peopleCache!.(personId)
  mailer.send! {
    subject = "hi"
    to = person.email
    body = "some details for you..."
  }

We resolve the peopleCache everytime we try to send an email to somebody: the first time we call peopleCache! we may have to wait for it to be populated. The second time, and all subsequent times, it will have been populated and it will return the cache immediately.

But what if Bob’s details take a long time to load, but Mary’s came back quickly: ideally we should still be able to send Mary an email. Let’s try a different approach and make each cache entry a promise:

detailsOfAllPeople () =
  {
    bob = personDetailsFor 'bob'
    mary = personDetailsFor 'mary'
  }

peopleCache = detailsOfAllPeople()

And then access the details as a promise:

sendEmailTo (personId) =
  person = peopleCache.(personId)!
  mailer.send {
    subject = "hi"
    to = person.email
    body = "some details for you..."
  }!

This way, peopleCache will be populated immediately but with promises of the poeple inside it. If one person’s details are slow to come back we can still send emails to the others.

In fact, let’s do it properly. Let’s put the caching behaviour into our personDetailsFor() function. That way the rest of our application doesn’t need to know about how we cache each person:

personDetailsFor (personId) =
  getPersonDetails() =
    httpism.get "http://api.app.com/people/#(personId)"!.body

  cached = peopleCache.(personId)
  if (cached)
    cached
  else
    peopleCache.(personId) = getPersonDetails()

With this function we look in the cache first and return the promise of the user’s details if it’s there. Otherwise we make a request for the user’s details, and put the promise into the cache.

One way to look at promises is that they are caches. Not only do they ensure (with !) that a value will eventually be there, once resolved, the value will always be there.

Timing

Promises aren’t just useful for getting information, they’re also useful in getting timing information. For example, we might not be interested in Mary’s details, we might be interested knowing when Mary’s details are ready.

From a UI perspective, we might want to indicate to the user that we have a person’s details, and that they can send them an email. We can disable the “email” button initially, and enable it when their details are ready.

button.disabled = true
personDetailsFor 'mary'!
button.disabled = false

Here we rely on caching a little bit: we’re making a request for Mary’s details even though we’re not interested in them. What we are interested in is the timing, we want to know when Mary’s details are ready so we can enable the button.

FIN

With a combination of the ! operator, promises and list comprehensions, writing reliable concurrent software in Pogoscript is very easy, so much so that we tend to do it all the time.

Testing with Mocha

Place your pogoscript specs in the test directory:

test/
  spec.pogo

Then invoke mocha with the --compilers switch:

mocha --compilers pogo:pogo

Or alternatively, place those arguments in test/mocha.opts and Mocha will use them as defaults:

--compilers pogo:pogo

You can place other arguments in there too, to select the spec reporter for example:

--compilers pogo:pogo --reporter spec

Promises

Testing asynchronous code in Mocha just works.

// test/spec.pogo

expect = require 'chai'.expect
express = require 'express'
httpism = require 'httpism'

describe 'api'
  api = nil
  server = nil

  beforeEach
    app = express()
    app.get '/' @(req, res)
      res.send { name = 'api', version = '2' }

    server := app.listen 12345

    api := httpism.api 'http://localhost:12345/'

  it 'can load root'
    expect (api.get '/'!.body).to.eql { name = 'api', version '1' }

Recent versions of Mocha support promises, so if you return a promise Mocha will wait for it to fulfill or be rejected. There’s no need to have a done parameter on your mocha test functions, just write normal pogoscript.