Quite some time ago, I added the capability for speech recognition (using pocketsphinx) and some rudimentary Zork-like natural language processing to Space Nerds In Space. However, like a dancing bear, the speech recognition is remarkable mostly in that works at all, not because it works so well. Especially, background noise is a big problem, but also accents, talking too fast, and mumbling slightly will make it not work so well.
The speech recognition is done in a separate process and the text which it produces is fed into the game via a fifo that snis_client monitors. This means the speech recognition can be replaced relatively easily (assuming the existence of an alternative). It occurred to me that if I had an android app which could do speech recognition (should be easy enough, right?) and then forward the recognized text to an arbitrary TCP socket, then I make some little program to forward this text from this TCP socket to the fifo. In fact, I don't even need to make something, I can just use netcat for that. However, I don't really know anything about writing android apps, and it's been almost 20 years since I even tried Java. If I were to do it, there's considerable static friction before I can get moving.
I mentioned this idea to Max Redmond, a member of my local hackerspace (these days, that's HackRVA) and hey presto, now we have an android app that does exactly what I imagined.
It's here:
Source code:
https://github.com/Athulus/SpaceNerdsCommunicatorReleases:
https://github.com/Athulus/SpaceNerdsCo ... r/releasesTo use this with SNIS, find out the IP address of one of the machines running snis_client, then
run this in a terminal:
- {l Code}: {l Select All Code}
netcat -lvk -p 8080 > /tmp/snis-natural-language-fifo
Then run the app on your android device, set the IP address, and then you can hit the microphone button and talk. Android will do the speech recognition (and do a much better job of it than pocketsphinx does) and the text will get forwarded to netcat and then into snis_client, and finally to snis_server, where it will be processed. It does not have the "Computer" hot word though.
It occurs to me now that unlike the fifo, the socket is a 2-way communications channel, so it might be worthwhile to set up a special socket for getting the NL text into snis_client and use that instead of or in addition to a fifo so that responses could be sent back to the android device as text (and then maybe text-to-speech). Then you could operate the ship from a phone remotely and at least get the responses from the computer instead of operating completely blind. Not that it would be super useful to try to play the game remotely via voice control without any visible output.
Perhaps a role-play mission where you're stranded on a planet, and you have to bring the unmanned ship into orbit around your planet using voice alone. i.e. "Set a course for planet X", "full speed ahead", "how far to planet X" (repeat until the reported number is small enough), "standard orbit", "beam me up scotty". Well, it's not much of a mission, I suppose.