All posts made in Jun. 2015:

More on conversational UIs

ICYMI, last week I dropped a ton of links + speculation on text messaging as user interface... Read it here. Alternatively catch up with:

  • Futures of text by Jonathan Libov of Union Square Ventures is a far, far better article than the one I wrote: A survey of all the current innovation in text as a medium. Plus: animated GIFs
  • Wired: The future of UI design? Old-school text messages is a quicker, more readable overview, and some neat extra points... It may always feel silly to talk out loud to Apple's virtual assistant; maybe Apple should let us text Siri instead.

I wanted to add a few more links.

Lark is a weight-loss coach that communicates with you exclusively through messaging.

Hello Lamp Post (detailed project page) is a playful SMS platform, inviting people to strike up conversations with familiar street furniture using the text message function of their mobile phones. Including escalating intimacy:

To help players feel as though their relationship with objects could develop, we built in a friendship mechanic - initial conversations would be a bit small-talky, about the weather and observations on the local environment, but on repeat visits the questioning of the objects would change, to focus on opinions, memories and beliefs.

(Unique qualities of text-based conversational UI... user-initiated conversations and app-initiation conversations feel the same, unlike regular apps; the element of time allows pauses and rhythms, like free-to-play games; it's how we already talk with our friends.)

Designing for text-based interfaces is going to take some experimentation.

What is conversation? is some decent theory... might be useful as a framework to talk about how conversations are structured and what's they're for. (Thanks @matt_thinkux.)

The word "just" creates a parent/child relationship. The article is in the context of women in the workplace, but this is an important point about language: Should a bot display deference? What should its stance be?

I'm definitely more into how all of this feels -- Alexis Lloyd (at the New York Times Research & Development group) wrote up her experiments: Our friends, the bots? I was curious to see what it would feel like to have a bot that was trying to engage as part of a social group

I haven't yet found the right words to characterize what this bot relationship feels like. It's non-threatening, but doesn't quite feel like a child or a pet. Yet it's clearly not a peer either. A charming alien, perhaps? The notable aspect is that it doesn't seem anthropomorphic or zoomorphic. It is very much a different kind of otherness, but one that has subjectivity and with which we can establish a relationship.


The conversation about how to define the bot's relationship to us really elucidated the idea that we are moving toward one member called "non-human mental models". We are beginning to understand machine subjectivity in a way that is in keeping with its nature rather than forcing it into other constructs, like a person or an animal.

This I love.

It's not just bots. How do we speak with non-humans, on their own terms? What does a bot want? Or a penguin, or a rock, or the military-industrial complex. Do we need human translators who can hold empathy for them on our behalf? Do we need a speaker for the thermocline? See also: The Author of the Acacia Seeds, Ursula K. Le Guin.

There's a hashtag used by speakers for the bots: #botALLY.


we are kind and gentle botmakers, allies to bots of all kinds and creeds

Found via that tag, a tool to help make Twitterbots: Cheap Bots, Done Quick!

e.g. @infinitedeserts, an infinity of deserts, each more infinite than the last.

(I'm no stranger to twitter bots, I made a presence machine and retold 99 Secrets -- both now silent.)

More on writing twitter bots, without code. More on writing twitter bots, with code.


Telegram Bot Platform. (Telegram is a messaging app with 60+ million monthly active users; it's growing fast.)

Bots are simply Telegram accounts operated by software - not people - and they'll often have AI features. They can do anything - teach, play, search, broadcast, remind, connect, integrate with other services, or even pass commands to the Internet of Things.

Neat about Telegram's approach, #1: Bots can now provide you with custom keyboards for specialized tasks (examples are shown). Any good bot platform is going to have to do this, typing is too cumbersome otherwise.

Neat about Telegram's approach, #2: any message from your bot forwarded to a person or group is a messaging equivalent of a retweet - bots are viral.

The really unique feature about conversational UIs is that messaging is social. Introductions can be made. Bots can take part in group conversations; facts can be remembered and shared. There's a figure and a ground.


Filtered for computers and birds


I... look... just... this super freaky image of squirrels is a picture drawn by a computer on its own.

This squirrel has a weird amount of eyes, yeah? And seems to be made at least partially of dogs? Check out its weird rear appendage, which is composed of slug tentacles that are themselves composed of birds. A two-headed fish lurks in the foreground, and upon reexamination the background is not mere swirls, but a warped, repetitive city

More of the backstory on the Google Research Blog.

It's a crazy process that works by turning up the gain on an artificial computer brain. Giving it acid...

We ask the network: "Whatever you see there, I want more of it!"" This creates a feedback loop: if a cloud looks a little bit like a bird, the network will make it look more like a bird. This in turn will make the network recognize the bird even more strongly on the next pass and so forth, until a highly detailed bird appears, seemingly out of nowhere.

Here's a gallery: Going deeper into neural networks.

Cloudscapes with hidden cities. Leaves that turn into birds and insects. Angels in the architecture.


There are so many things to love about Paul Ford's epic Businessweek-takeover essay What is Code? but I'm taken with his insistence on treating the computer as a physical thing -- which is of course it is, all teeny-weeny electrons and stuff...

What are the steps and layers between what you're doing and the Lilliputian mechanisms within?

Just as the keyboard is waiting for a key to be pressed, the computer is waiting for a signal from the keyboard. When one comes down the pike, the computer interprets it and passes it farther into its own interior.


What do they call turkeys in Turkey?

Answer: Hindi.

In Hindi, a turkey is called Peru, which is itself a borrowing from the Portuguese.

The Language of the Birds is an angelic language, a divine language which predates and supersedes human speech, used by birds, understood by the initiated.

There's a funny-peculiar reference in Understanding Media (Marshall McLuhan) to Jason's quest for the Golden Fleece, and his sowing of dragon's teeth which grew to become the spear-tips of a new army. McLuhan enlists the legend of the dragon's teeth as an allegory for the development of the alphabet and, at the same time, bodies of armed men.

Saying thank you in Hindi destroys intimacy.


Data furnaces:

Microsoft has a lot of servers, mostly sitting in large data centres, producing huge amounts of heat -- heat that is a massive nuisance to deal with. ... Instead of finding a novel way of transferring waste heat away from the data centre, the research paper proposed that the servers themselves should be placed in homes and offices, where the waste heat could be used directly.

Data furnaces arrive in Europe: Free heating, if you have fibre Internet. Nerdalize is rolling out eRadiators in the Netherlands, providing 1000W of heat.

On conversational UIs


When trumpets were mellow

And every gal only had one fellow

No need to remember when

'Cause everything old is new again

-- Peter Allen.

Stand back folks. I've not spent any time editing and now I'm going out. This is stream of consciousness, and it's long.


There's that bit in the great article on Chinese mobile UI trends about how there are no websites, there's just messaging. And not only that, some weird mish-mash of talking robots and customer service people:

Many institutions that otherwise would have native apps or mobile sites have opted instead for official accounts. You can send any kind of message (text, image, voice, etc), and they’ll reply, either in an automated fashion or by routing it to a human somewhere. The interface is exactly the same as for chatting with your friends

You know, and why the hell not. I have one language to use with apps (pointing, tapping, swiping) and another with my friends (chatting). Why not chat with my apps too?

So as Benedict Evans - mobile and technology analyst extraordinaire - points out, messaging is the new app platform:

[In WeChat, in China] You can send money, order a cab, book a restaurant or track and manage an ecommerce order, all within one social app. So, like the web, you don't need to install new apps to access these services, but, unlike the web, they can also use push and messaging and social to spread.

The other piece of the puzzle here, Evans continues, is the smartphone notifications panel:

That pull-down panel aggregates activity from everything on your phone, and Google and Apple have made notifications actionable and given them payloads. ... More and more, one's primary interaction with any app, social messaging or otherwise, is a little pop-up with a button or two.

So I've long been interested in the idea that "next actions" should float away from their apps and come together in a single place... SNAP was my 2008 take on this.

But I guess the 2015 twist is that everything old is new again, and we're dealing not just with actionable notifications, but robot-generated text that we can have an actual conversation with.

Which is Twitter's fault.

Nowhere is it more evident that The Web is a Customer Service Medium. Just look at @AskNationwide (with replies) or @British_Airways (with replies). It's all conversations with customers.

The canonical automated version of this is @andy_house (2009), aka the house that twitters, or see House of Coates for a more up-to-date take on the tweeting smart home.

Now imagine it wasn't just an activity feed, but you could talk back.

A big bit of the current excitement is the rise of Slack for workplace comms and its embrace of bots. Which takes us to Ben Brown's insanely incredible insight: What happens when you start automating workplace processes?

What if there was a meeting runner bot that automatically sent out an agenda to all attendees before the meeting, then collected, collated and delivered updates to team members? It could make meetings shorter and more productive by reducing the time needed to bring everyone up to speed.

We've just been through an era where management has been regarded as the essential scarce resource of a business, and operations and technology are functions to be outsourced to fungible workers like so many cogs. But what if the core business resource is human ingenuity, and it's management that can be turned into software... automated and optimised?


Digit is an automated savings tool: Every few days, Digit checks your spending habits and removes a few dollars from your checking account if you can afford it.

The kicker: You communicate with it via text message ("Great, I've moved $10.00 to digit"), they have no plans for an app. And what's interesting to me is that it has adaptive behaviour... and maybe because of the text message interface, this Digit review semi-anthropomorphises the software:

At first, Digit was really cautious with my money ... But over the next couple weeks, as my balance recovered from holiday spending, it got a bit more ambitious

Software isn't "cautious" or "ambitious", those are qualities of alive beings. But maybe it serves us to think so.

Walkadoo is a walking game that encourages activity; you communicate with it by text message. Related: Autom is a robot weight-loss coach with big blue eyes. You lose more weight because you regard it as having a mind.

Rhombus is an e-commerce platform for shop to chat with customers by text message... and also accept payments, within the conversation. Related: Twitter's in-feed Buy Now button which is a game changer if widely rolled out.


One of the problems with text interfaces is text entry. Keyboards suck, especially on mobile devices. Typing also introduces a discoverability question: How do you know what words are valid right now, or the right grammar to use? How do you make complex statements?

In the game Lifeline (iPhone/Apple Watch) you're texting an astronaut called Taylor who is marooned on a moon. It works in real-time... when Taylor hikes to a location an hour away, you won't hear from her till she gets there. You text back, but it's not free text entry, you only get two options at a time. Works well on the smartwatch too.

Despite the constrained responses, it still feels conversational. Enough that the first time I killed Taylor - by freezing to death based on advice I'd given - well, ooof.

Lifeline was prototyped in Twine, the visual programming language for writing interactive fiction. See also Inform 7 where the code-behind-the-fiction reads like a book but every word has its code-meaning too, like casting a spell in a stupid universe, like talking to a golem. e.g.:

The dark doorway is an open door. "The doorway looks ordinary enough, but it's so difficult to be sure with the unaided eye." The dark doorway is scenery. The dark doorway is not openable. The dark doorway is west of Longwall Street and east of Turret Roundhouse. The dark doorway is trapped.

Another take on text input:

The web-based game A Dark Room (also available for iPhone) is astounding. Half text adventure, half point-and-click. There's something about clicking stoke fire and the words turning into a progress bar while the fire burns down... The communication of the element of time.

Hangkeys is a neat hack -- it's a custom iPhone keyboard that makes it super easy to play Hangman over SMS, Whatsapp, or other text message services.

Meet by Sunrise is a custom smartphone keyboard that integrates with your calendar: Instead of typing times and locations, you tap them instead.

Matt Galligan imagines tap-able buttons in text messages: What if instead of installing an app, we might instead allow a service to chat with us via iMessage?

Writing code... Swift allows emoji for variable names. There's something interesting about this. Variable names like "a" or "theCounter" or "dimensions" are meaningful... but what about underlying feelings they carry? The "ii" counter of a tight inner loop always has a zing for me, it's the twang of high-tension power lines.

So can emoji carry more meaning, or meaning along a different axis? What could we use this for -- instead of "houseAddress" just have a picture of a house; instead of saying errorDescription just use smiling pile of poo emoji.


I was noodling with conversational UIs in 2002, back when AIM was a thing. What preoccupied me then - and what interests me most now - is how to make an automated conversation with a bot not boring, and (more importantly) not shit.

So one of the things I found before: The more the bot acts like a human, the more it will be treated like a human. And the more that happens, the more likely the bot will have to say "I don't know what you mean"... which is lame.

Our guiding light is The Jack Principles from the 1995 quiz show video game You Don't Know Jack. In short, how to direct the conversation so a user will never be in a position to ask a question the machine can't answer.

(Tom Armitage collected a link to this and more when he ran a chatbot project at Berg back in 2010.)

Something else surprised me about authoring conversations, something almost contradictory...

  • Leading a user through a conversation by the hand is great... once. It's like a phone tree: a branching tree of questions and multiple choice answers where nothing can go wrong. But if the user wants to change their answer, compare different possibilities, or run through the conversation a second time: It's tedious and frustrating.
  • But the alternative is a wide open space: The user can say anything, and it's up to the bot to interpret and respond. However that introduces a discoverability problem. You can be chatting to your bot about what's on TV tonight, completely unaware that it also knows what movies are showing nearby. Or let's say you do ask about movies, and then you say "well how do I get to the theatre?" and suddenly it's dumb again. Using Siri is often like that.

I guess what I'm asking is how does a user have a theory of mind about a bot - a conception of its stance, intentions, domain of knowledge, etc - and how is that communicated.

My take back in the day was to organise knowledge into domains, within which a tree structure would be possible but avoided. To summarise how it worked:

  • I built an AIM bot called FingertipTV. You could traverse it like a tree: "now" (reply: "a list of channels and what's on now"), "bbc1" (reply: now and next for BBC1), "9pm" (reply: what's on BBC1 at 9pm and the following show)
  • Resolution varied. "now" would result in a list of every channel but just the show titles (and you could hit a number to get the full description). Getting more precise with the channel and "9pm" would show a full show description.
  • Once you'd learnt the vocabulary, everything allowed shortcuts: The user could gradually become virtuoso. So the previous exchange could be replaced with "bbc1 9pm"
  • There was a limited amount of history. So saying "bbc1" then "later" would show what was on BBC1 later... but saying "bbc1", "9pm", "channel4" would show Channel 4 right now. The 9pm itself was forgotten. Use any excuse to limit the memory of the bot, because that constrains what the user has to hold in mind
  • I added functionality to search movie listings... but encapsulated that in another AIM bot called Cinebot. If you asked about movies to FingertipTV, you'd get the reply "I can't help you but my friend can," and Cinebot would reach out and start a new conversation. Now you'd met Cinebot, and could add their name to your buddy list. Small, understandable, almost-stateless bots that talk to you and to each other

My point, I guess, is that a new medium needs a new grammar and conversational UIs are definitely a new medium.

For one -- they're intrinsically social. If I'm chatting with a bot in iMessage about what movies are on nearby, shouldn't I be able to turn that into a group chat with my partner? And does the bot conduct two separate conversations, one with each of us, or assume we're both searching for the same movie?

We'll need app frameworks to help author these bots, and the frameworks will make assumptions about how conversations work in small groups.

And you know what, we're still in the PC era, the era of personal computing. We don't really know how to use computers in small groups, how to use interfaces collaboratively.

Another big question for me is what happens when we have many of these bots...

How do traverse multiple knowledge domains, discovering features and adapting how we speak as we go? Is it going to be like FingertipTV and Cinebot, loosely coupled domains of differing knowledge and vocabulary? Or more like Siri where the same voice can tell you everything from "directions home" to "how long do dogs live" (to pick two examples that it's giving me right now).

Maybe it will be like Twitter, where everyone I follow is in a single stream -- but I miss what they're saying? Or like my apps on my phone, where many apps have their own activity stream... and they fight so hard to get heard that they constantly spam my notifications panel?

My head goes somewhere quite speculative,

and that's text adventures.

Dan's story, Text Only:

What now? >


You go North

The batteries on your Discman are almost depleted.

Or Julian Dibbell's memoire My Tiny Life, a tale of LambdaMoo, a collaboratively built text environment inhabited by objects, bots, and people.

There's a strength in spatialising information -- of arranging these bots - these domains of knowledge and differing patterns of interaction - into a web, or on a map. Some bots are closer to other bots.

So part of me wonders... what if I saw the activity feed from my smart home all together in one place, and then when I went north, say, that would be the activity feed from my social networks. Or instead, in my smart home I'd find my TV, and inside that we could have a chat about what's being tivo'd tonight.

I don't want to be too literal. But maybe we need an architecture to arrange all these bots and feeds and conversations, etc. And while our experience of the conversations will vary (we'll be friends with different bots) the architecture will be shared: Arbitrary, but shared. A cyberspace:

A consensual hallucination experienced daily by billions of legitimate operators ... [a] representation of data abstracted from banks of every computer in the human system.


And don't throw the past away

You might need it some other rainy day

Dreams can come true again

When everything old is new again

When everything old is new again

I might fall in love with you again


ps. More on conversational UIs.

Filtered for lists


Pixar's 22 rules of storytelling. e.g.:

  • You admire a character for trying more than for their successes.
  • When you're stuck, make a list of what WOULDN'T happen next. Lots of times the material to get you unstuck will show up.

Reminds me of Michael B. Johnson talking about Pixar's process...

  • Make a believable world
  • Create a character that is believable in that world
  • Create a story
  • Repeat until done

I seem to remember there was a precedence ordering to this. Like, the believable characters in the Toy Story world are toys, right, so they don't have tear ducts. Which means you can't have them cry, no matter how much the story demands it -- you have to show that emotion some other way, perhaps with rain and clever camera work. World > character > story.


16 Perfect Japanese Words You Need In Your Life.

Tsundoku. The act of buying a book and leaving it unread

Yugen. A profound awareness of the universe that triggers a deep emotional response


General Orders for Sentries.

On the surface, it's a pretty simple thing, being a sentry. "Watch this area. Tell us if anything odd happens."

In practice, there's thousands of permeations of things that can go wrong when you're guarding an area, and a sentry failing in his duty could lead to -- literally -- thousands of people getting killed.

And so there is a list of 11 rules - short enough to memorise - that covers what to do. The Three Laws of Robotics only for soldiers.

Reading these rules, I have a picture of the military as a machine -- or rather as agent-based hardware, and these rules are the software. In a sentry role? Load the General Orders for Sentries.

It's the playbook.

(I've been writing a playbook recently... just for a project I'm sketching out. Keeps me out of trouble. It's interesting to think of a project or "business" as a platform of people and technology, running "software" made of actual code and of playbooks. Included in the "software" is the capacity for learning and self-healing. The playbook/software I'm sketching is made of scheduled triggers, plays which are like checklists or state machines, and pipelines. In the future, business bureaucracy will be automated -- it'll turn into software, just like everything else that doesn't require human ingenuity or human dexterity.)


Animals that do not synthesise their own vitamin C:

  • Fruit bats
  • Guinea pigs
  • Primate monkeys
  • Humans

Our word for the colour orange comes from the name of the fruit; the first recorded use was in 1512.

Filtered for radioactive cats


Question. If you bury radioactive waste and need to warn people to stay away from the land for 10,000 years, how do you do it -- basically how do you construct a message that lasts longer than humans have been living in cities?

Answer from 1981: cats and songs.

Bastide and Fabbri came to the conclusion that the most durable thing that humanity has ever made is culture: religion, folklore, belief systems. They may morph over time, but an essential message can get pulled through over millennia. They proposed that we genetically engineer a species of cat that changes color in the presence of radiation, which would be released into the wild to serve as living Geiger counters. Then, we would create folklore and write songs and tell stories about these "ray cats," the moral being that when you see these cats change colors, run far, far away.


The Green Man is often related to natural vegetative deities.

The Green Man appears in many forms, with the three most common types categorized as: the Foliate Head: completely covered in green leaves; the Disgorging Head: spews vegetation from its mouth; the Bloodsucker Head: sprouts vegetation from all facial orifices (e.g. tear ducts, nostrils and mouth)


Traditional rhymes for counting sheep vary region by region. This is how to count to 10 in Derbyshire:

  • Yan
  • Tan
  • Tethera
  • Methera
  • Pip
  • Sethera
  • Lethera
  • Hovera
  • Dovera
  • Dick


Greg Egan - author of the hardest of hard sci-fi - has a particular take on science fiction:

I'm interested in science as a subject in its own right, just as much as I'm interested in the effects of technology on the human condition. In many things I write the two will be combined, but even then it's important to try to describe the science accurately. In a novel such as Incandescence, though, the entire point is understanding the science, and it really doesn't bother me in the least that it's not an exploration of the human condition.

There are times when it's worth putting aside the endless myopic navel-gazing that occupies so much literature, in order to look out at the universe itself and value it for what it is.

Fuck yeah. Incandescence is a crazy novel because - although it's set up by the short story Riding the Crocodile which is a great story in its own right - it barely has any story: It's really about the discovery of physics and its implications from the perspective of a society living (unknowingly) close to a black hole, in the realm where General Relativity dominates over Newtonian physics.

It's hard going, but it's a book of unfolding implications. Like Neal Stephenson's new novel Seveneves (review) which is barely a story, and more like a rigorous proof-by-deduction of what would happen if the Moon blew up.

What I love about this - aside from the de-pedestaling of story, and managing to escape that particular tyranny - is the novels can walk you into a deep understanding of accurate-but-nonintuitive natural systems. It seems to me that I think by applying metaphors - by pattern recognition - this is how the world works, this is what feels familiar. Physics gives me a richer set of metaphors: I believe I'm a stronger thinker for having encountered Cooper pairs. Post Seveneves I find myself applying the metaphor of orbital mechanics to, let's say, personal brand -- just idly, not rigorously, but Stephenson's novel has given me the beginnings of a new intuition to test out.

And there's a joy in the process of learning and encountering. So why shouldn't science fiction be fiction about science and the scientific method itself? Egan's bang-on, even if it's at times difficult. Or honestly: blimmin tedious... but in a good way. More power to his elbow.

Filtered for heavy dogs


Tinitell, a wristphone for kids.

Product video.

Voice recognition, a battery that lasts a week, kid-to-kid comms, and GPS for the parents.


Keecker, a video projector attached a home robot.

Some interesting use-cases...

  • walk through your house skyping with a friend in Tokyo with KEECKER following you,
  • or see what's happening in real time by moving KEECKER around your house [I assume remotely]

I mean, my Apple Watch means notifications follow me around by virtue of it being tied to my wrist. But another way of doing this is computation literally following me around.

See also the Lily drone -- a drone with follow-me functionality, and 20 minutes of HD video. I love the idea that the language of film direction is coming into everyday life, that you could call out to your Lily different camera angles at it swings around you snowboarding down a mountain.

Or walking to the shops, whatever.


Tingbot, Raspberry Pi-powered alarm-clock-form-factor box to run apps at home.


From New Scientist, February 1976:

A US nuclear laboratory is said to maintain a "heavy dog"--fed with heavy water and "heavy" food, and with the heavy isotope deuterium replacing normal hydrogen throughout the whole body.

The heavy dog would have some unusual properties,

[it] would be invisible to a MRI scanner. The dog would still be clearly visible to the naked eye so apart from confusing the staff operating the scanner I can see no advantage to this.

Heavy dog.