Skip to content

Commit

Permalink
Added snowboy folder, and new authors script for updating authors fro…
Browse files Browse the repository at this point in the history
…m git log.
  • Loading branch information
chrobione committed Aug 4, 2016
1 parent f7fdc26 commit ae3385d
Show file tree
Hide file tree
Showing 22 changed files with 5,466 additions and 20 deletions.
54 changes: 34 additions & 20 deletions AUTHORS.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,37 @@
Jasper was originally created by Shubhro Saha and Charles Marsh in early 2014.
In August 2014, the core development team has been joined by Jan Holthuis.
# Authors

Jan Holthuis <[email protected]>
Charles Marsh <[email protected]>
Shubhro Saha <[email protected]>
#### Ordered by first contribution.

Here is a probably incomplete list of contributors, that helped to improve
Jasper. Thanks a lot!
- Jasper Project <[email protected]>
- Shubhro Saha <[email protected]>
- Charles Marsh <[email protected]>
- David Celis <[email protected]>
- Alex Bate <[email protected]>
- Charles Marsh <[email protected]>
- Andy Buckingham <[email protected]>
- Daniel Geymayer <[email protected]>
- dag0310 <[email protected]>
- Andrew Stahlman <[email protected]>
- Andrew Stahlman <[email protected]>
- Alex <[email protected]>
- Jan Holthuis <[email protected]>
- Jan Holthuis <[email protected]>
- alexsiri7 <[email protected]>
- James Timmons <[email protected]>
- Kaleb Pomeroy <[email protected]>
- Cedric Claidiere <[email protected]>
- x-hamiltonian <[email protected]>
- Nick Albright <[email protected]>
- Jean-Francois Remy <[email protected]>
- Steven Hiscocks <[email protected]>
- Ben Hoff <[email protected]>
- Jean-François Remy <[email protected]>
- Edouard Poitras <[email protected]>
- Sytone <[email protected]>
- ThomasDelteil <[email protected]>
- Andreas Weber <[email protected]>
- Dan Buchta <[email protected]>
- Sebastien <[email protected]>
- chrobione <[email protected]>

Alex Bate <[email protected]>
Andy Buckingham <[email protected]>
David Celis <[email protected]>
Cedric Claidiere
dag0310
Alex Siri
Andrew Stahlman <[email protected]>
James Timmons

*Please alphabetize new entries*

We'd also like to thank all the people who reported bugs, helped
answer newbie questions, and generally made Jasper better.
#### Generated by bin/update-authors.sh.
23 changes: 23 additions & 0 deletions AUTHORS.md.org
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
Jasper was originally created by Shubhro Saha and Charles Marsh in early 2014.
In August 2014, the core development team has been joined by Jan Holthuis.

Jan Holthuis <[email protected]>
Charles Marsh <[email protected]>
Shubhro Saha <[email protected]>

Here is a probably incomplete list of contributors, that helped to improve
Jasper. Thanks a lot!

Alex Bate <[email protected]>
Andy Buckingham <[email protected]>
David Celis <[email protected]>
Cedric Claidiere
dag0310
Alex Siri
Andrew Stahlman <[email protected]>
James Timmons

*Please alphabetize new entries*

We'd also like to thank all the people who reported bugs, helped
answer newbie questions, and generally made Jasper better.
Binary file added plugins/stt/snowboy-stt/snowboy/Jasper.pmdl
Binary file not shown.
Binary file not shown.
55 changes: 55 additions & 0 deletions plugins/stt/snowboy-stt/snowboy/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Example Makefile that converts snowboy c++ library (snowboy-detect.a) to
# python library (_snowboydetect.so, snowboydetect.py), using swig.

# Some versions of swig does not work well. We prefer compiling swig from source
# code. We have tested swig-3.0.7.tar.gz.
SWIG := swig

SNOWBOYDETECTSWIGITF = snowboy-detect-swig.i
SNOWBOYDETECTSWIGOBJ = snowboy-detect-swig.o
SNOWBOYDETECTSWIGCC = snowboy-detect-swig.cc
SNOWBOYDETECTSWIGLIBFILE = _snowboydetect.so

TOPDIR := ../../
CXXFLAGS := -I$(TOPDIR) -O3 -fPIC
LDFLAGS :=

ifeq ($(shell uname), Darwin)
CXX := clang++
PYINC := $(shell /usr/bin/python2.7-config --includes)
PYLIBS := $(shell /usr/bin/python2.7-config --ldflags)
SWIGFLAGS := -bundle -flat_namespace -undefined suppress
LDLIBS := -lm -ldl -framework Accelerate
SNOWBOYDETECTLIBFILE = $(TOPDIR)/lib/osx/libsnowboy-detect.a
else
CXX := g++
PYINC := $(shell python-config --cflags)
PYLIBS := $(shell python-config --ldflags)
SWIGFLAGS := -shared
CXXFLAGS += -std=c++0x
# Make sure you have Atlas installed. You can statically link Atlas if you
# would like to be able to move the library to a machine without Atlas.
LDLIBS := -lm -ldl -lf77blas -lcblas -llapack_atlas -latlas
SNOWBOYDETECTLIBFILE = $(TOPDIR)/lib/ubuntu64/libsnowboy-detect.a
ifneq (,$(findstring arm,$(shell uname -m)))
SNOWBOYDETECTLIBFILE = $(TOPDIR)/lib/rpi/libsnowboy-detect.a
endif
endif

all: $(SNOWBOYSWIGLIBFILE) $(SNOWBOYDETECTSWIGLIBFILE)

%.a:
$(MAKE) -C ${@D} ${@F}

$(SNOWBOYDETECTSWIGCC): $(SNOWBOYDETECTSWIGITF)
$(SWIG) -I$(TOPDIR) -c++ -python -o $(SNOWBOYDETECTSWIGCC) $(SNOWBOYDETECTSWIGITF)

$(SNOWBOYDETECTSWIGOBJ): $(SNOWBOYDETECTSWIGCC)
$(CXX) $(PYINC) $(CXXFLAGS) -c $(SNOWBOYDETECTSWIGCC)

$(SNOWBOYDETECTSWIGLIBFILE): $(SNOWBOYDETECTSWIGOBJ) $(SNOWBOYDETECTLIBFILE)
$(CXX) $(CXXFLAGS) $(LDFLAGS) $(SWIGFLAGS) $(SNOWBOYDETECTSWIGOBJ) \
$(SNOWBOYDETECTLIBFILE) $(PYLIBS) $(LDLIBS) -o $(SNOWBOYDETECTSWIGLIBFILE)

clean:
-rm -f *.o *.a *.so snowboydetect.py *.pyc $(SNOWBOYDETECTSWIGCC)
197 changes: 197 additions & 0 deletions plugins/stt/snowboy-stt/snowboy/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
# Snowboy Hotword Detection

by [KITT.AI](http://kitt.ai).

[Home Page](https://snowboy.kitt.ai)

[Full Documentation](https://snowboy.kitt.ai/docs)


Version: 1.0.4 (7/13/2016)

Snowboy is a customizable hotword detection engine for you to create your own
hotword like "OK Google" or "Alexa". It is powered by deep neural networks and
has the following properties:

* **highly customizable**: you can freely define your own magic phrase here –
let it be “open sesame”, “garage door open”, or “hello dreamhouse”, you name it.

* **always listening** but protects your privacy: Snowboy does not use Internet
and does *not* stream your voice to the cloud.

* light-weight and **embedded**: it even runs on a Raspberry Pi and consumes
less than 10% CPU on the weakest Pi (single-core 700MHz ARMv6).

* Apache licensed!

Currently Snowboy supports:

* all versions of Raspberry Pi (with Raspbian based on Debian Jessie 8.0)
* 64bit Mac OS X
* 64bit Ubuntu (12.04 and 14.04)
* iOS
* Android

It ships in the form of a **C++ library** with language-dependent wrappers
generated by SWIG. We welcome wrappers for new languages -- feel free to send a
pull request!

If you want support on other hardware/OS, please send your request to
[[email protected]](mailto:snowboy.kitt.ai)

## Precompiled Binaries with Python Demo
* 64 bit Ubuntu [12.04](https://s3-us-west-2.amazonaws.com/snowboy/snowboy-releases/ubuntu1204-x86_64-1.0.4.tar.bz2)
/ [14.04](https://s3-us-west-2.amazonaws.com/snowboy/snowboy-releases/ubuntu1404-x86_64-1.0.4.tar.bz2)
* [MacOS X](https://s3-us-west-2.amazonaws.com/snowboy/snowboy-releases/osx-x86_64-1.0.4.tar.bz2)
* Raspberry Pi with Raspbian 8.0, all versions
([1/2/3/Zero](https://s3-us-west-2.amazonaws.com/snowboy/snowboy-releases/rpi-arm-raspbian-8.0-1.0.4.tar.bz2))

If you want to compile a version against your own environment/language, read on.

## Dependencies

Snowboy's Python wrapper uses PortAudio to access your device's microphone.

### Mac OS X

`brew` install `swig`, `sox`, `portaudio` and its Python binding `pyaudio`:

brew install swig portaudio sox
pip install pyaudio

If you don't have Homebrew installed, please download it [here](http://brew.sh/). If you don't have `pip`, you can install it [here](https://pip.pypa.io/en/stable/installing/).

Make sure that you can record audio with your microphone:

rec t.wav

### Ubuntu/Raspberry Pi

First `apt-get` install `swig`, `sox`, `portaudio` and its Python binding `pyaudio`:

sudo apt-get install swig3.0 python-pyaudio python3-pyaudio sox
pip install pyaudio

Then install the `atlas` matrix computing library:

sudo apt-get install libatlas-base-dev

Make sure that you can record audio with your microphone:

rec t.wav
If you need extra setup on your audio (especially on a Raspberry Pi), please see the [full documentation](https://snowboy.kitt.ai/docs).

## Compile a Python Wrapper

cd swig/Python
make

SWIG will generate a `_snowboydetect.so` file and a simple (but hard-to-read) python wrapper `snowboydetect.py`. We have provided a higher level python wrapper `snowboydecoder.py` on top of that.

Feel free to adapt the `Makefile` in `swig/Python` to your own system's setting if you cannot `make` it.


## Compile an iOS Wrapper

Using Snowboy library in Objective-C does not really require a wrapper. It is basically the same as using C++ library in Objective-C. We have compiled a "fat" static library for iOS devices, see the library here `lib/ios/libsnowboy-detect.a`.

To initialize Snowboy detector in Objective-C:

snowboy::SnowboyDetect* snowboyDetector = new snowboy::SnowboyDetect(
std::string([[[NSBundle mainBundle]pathForResource:@"common" ofType:@"res"] UTF8String]),
std::string([[[NSBundle mainBundle]pathForResource:@"snowboy" ofType:@"umdl"] UTF8String]));
snowboyDetector->SetSensitivity("0.45"); // Sensitivity for each hotword
snowboyDetector->SetAudioGain(2.0); // Audio gain for detection

To run hotword detection in Objective-C:

int result = snowboyDetector->RunDetection(buffer[0], bufferSize); // buffer[0] is a float array

You may want to play with the frequency of the calls to `RunDetection()`, which controls the CPU usage and the detection latency.

## Compile an Android Wrapper

cd swig/Android
# Make sure you set up the NDKROOT variable in Makefile before you run.
# We have only tested with NDK version r11c.
make

Using Snowboy library on Android devices is a little bit tricky. We have only tested with NDK version r11c. We do not support r12 yet because of the removal of armeabi-v7a-hard ABI in r12. We have compiled Snowboy using Android's cross-compilation toolchain for ARMV7 architecture, see the library here `lib/android/armv7a/libsnowboy-detect.a`. We then use SWIG to generate the Java wrapper, and use Android's cross-compilation toolchain to generate the corresponding JNI libraries. After running `make`, two directories will be created: `java` and `jniLibs`. Copy these two directories to your Android app directory (e.g., `app/src/main/`) and you should be able to call Snowboy funcitons within Java.

To initialize Snowboy detector in Java:

# Assume you put the model related files under /sdcard/snowboy/
SnowboyDetect snowboyDetector = new SnowboyDetect("/sdcard/snowboy/common.res",
"/sdcard/snowboy/snowboy.umdl");
snowboyDetector.SetSensitivity("0.45"); // Sensitivity for each hotword
snowboyDetector.SetAudioGain(2.0); // Audio gain for detection

To run hotword detection in Java:

int result = snowboyDetector.RunDetection(buffer, buffer.length); // buffer is a short array.

You may want to play with the frequency of the calls to `RunDetection()`, which controls the CPU usage and the detection latency.

## Quick Start for Python Demo

Go to the `examples/Python` folder and open your python console:

In [1]: import snowboydecoder

In [2]: def detected_callback():
....: print "hotword detected"
....:

In [3]: detector = snowboydecoder.HotwordDetector("resources/snowboy.umdl", sensitivity=0.5, audio_gain=1)

In [4]: detector.start(detected_callback)

Then speak "snowboy" to your microphone to see whetheer Snowboy detects you.

The `snowboy.umdl` file is a "universal" model that detect different people speaking "snowboy". If you want other hotwords, please go to [snowboy.kitt.ai](https://snowboy.kitt.ai) to record, train and downloand your own personal model (a `.pmdl` file).

When `sensitiviy` is higher, the hotword gets more easily triggered. But you might get more false alarms.

`audio_gain` controls whether to increase (>1) or decrease (<1) input volume.

Two demo files `demo.py` and `demo2.py` are provided to show more usages.

Note: if you see the following error:

TypeError: __init__() got an unexpected keyword argument 'model_str'

You are probably using an old version of SWIG. Please upgrade. We have tested with SWIG version 3.0.7 and 3.0.8.

## Advanced Usages & Demos

See [Full Documentation](https://snowboy.kitt.ai/docs).

## Change Log

**v1.0.4, 7/13/2016**

* Updated universal `snowboy.umdl` model to make it more robust.
* Various improvements to speed up the detection.
* Bug fixes.

**v1.0.3, 6/4/2016**

* Updated universal `snowboy.umdl` model to make it more robust in non-speech environment.
* Fixed bug when using float as input data.
* Added library support for Android ARMV7 architecture.
* Added library for iOS.

**v1.0.2, 5/24/2016**

* Updated universal `snowboy.umdl` model
* added C++ examples, docs will come in next release.

**v1.0.1, 5/16/2016**

* VAD now returns -2 on silence, -1 on error, 0 on voice and >0 on triggered models
* added static library for Raspberry Pi in case people want to compile themselves instead of using the binary version

**v1.0.0, 5/10/2016**

* initial release
1 change: 1 addition & 0 deletions plugins/stt/snowboy-stt/snowboy/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@

35 changes: 35 additions & 0 deletions plugins/stt/snowboy-stt/snowboy/demo.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
import snowboydecoder
import sys
import signal

interrupted = False


def signal_handler(signal, frame):
global interrupted
interrupted = True


def interrupt_callback():
global interrupted
return interrupted

if len(sys.argv) == 1:
print("Error: need to specify model name")
print("Usage: python demo.py your.model")
sys.exit(-1)

model = sys.argv[1]

# capture SIGINT signal, e.g., Ctrl+C
signal.signal(signal.SIGINT, signal_handler)

detector = snowboydecoder.HotwordDetector(model, sensitivity=0.5)
print('Listening... Press Ctrl+C to exit')

# main loop
detector.start(detected_callback=snowboydecoder.play_audio_file,
interrupt_check=interrupt_callback,
sleep_time=0.03)

detector.terminate()
Loading

0 comments on commit ae3385d

Please sign in to comment.