WebOS’s Weekly App Hack 10-05-11

Welcome from the Palm Developer blog! Looking for binaries?

Having trouble posting your app or an update to your app using the provided sdltts? Either you’re missing features or you want/need a callback from the method when it’s done, or maybe you want only a few text strings in your app and don’t want to keep generating them? If you use the provided sdltts, there isn’t much you can do about most of your problems.

So I worked a little on re-creating the binary myself, I like doing these sort of exploratory projects. I’ll put the diff and necessary scripts below, but first, here’s the status:

On Touchpad:

  • Runs just like the given binary when used in their provided app in place of the sdltts binary.
  • Update 10/12/11: Runs in separate thread, thanks to unwiredben.

On Emulator:

  • WORKS inside the application. (As of 10/9/11 11:48AM)

Code

Download the library’s source for release 1.4 and make these changes.

main/Makefile diff:

diff -ur flite-1.4-release/main/Makefile flite-1.4-webos/main/Makefile
--- flite-1.4-release/main/Makefile     2009-08-14 14:11:07.000000000 -0700
+++ flite-1.4-webos//main/Makefile      2011-10-08 01:21:54.012426846 -0700
@@ -38,13 +38,13 @@
 DIRNAME=main
 BUILD_DIRS =
 ALL_DIRS=
-SRCS = flite_main.c flite_time_main.c t2p_main.c compile_regexes.c
+SRCS = flite_main.c sdltts.c flite_time_main.c t2p_main.c compile_regexes.c
 OBJS = $(SRCS:.c=.o) flite_voice_list.o
 FILES = Makefile $(SRCS)
 LOCAL_INCLUDES =

 ALL = shared_libs \
-      $(BINDIR)/flite$(EXEEXT) \
+      $(BINDIR)/sdltts$(EXEEXT) \
       $(BINDIR)/t2p$(EXEEXT) $(BINDIR)/compile_regexes$(EXEEXT) \
       flite_voice_list.c each $(EXTRABINS)

@@ -70,6 +70,7 @@
 flite_time_LIBS_deps = $(flite_time_LIBS:%=$(LIBDIR)/lib%.a)

 LOCAL_CLEAN = $(BINDIR)/flite$(EXEEXT) $(BINDIR)/flite_time$(EXEEXT) \
+              $(BINDIR)/sdltts$(EXEEXT) \
               $(BINDIR)/t2p$(EXEEXT) \
               $(SHAREDARLIBS) $(SHAREDLIBS) $(VERSIONSHAREDLIBS) \
               $(flite_LIBS_deps) $(VOICES:%=$(BINDIR)/flite_%)
@@ -81,6 +82,12 @@
 shared_libs: nothing
 endif

+$(BINDIR)/sdltts$(EXEEXT): sdltts.o $(flite_LIBS_deps)
+       $(TOP)/tools/make_voice_list $(VOICES)
+       rm -f flite_voice_list.o
+       $(MAKE) flite_voice_list.o
+       $(CC) $(CFLAGS) -o $@ sdltts.o flite_voice_list.o $(flite_LIBS_flags) $(LDFLAGS)
+
 $(BINDIR)/flite$(EXEEXT): flite_main.o $(flite_LIBS_deps)
        $(TOP)/tools/make_voice_list $(VOICES)
        rm -f flite_voice_list.o
@@ -103,11 +111,11 @@
$(MAKE) VOICE=$$i $(BINDIR)/flite_$$i ; \
done

-$(BINDIR)/flite_${VOICE}: flite_main.o $(flite_LIBS_deps)
+$(BINDIR)/flite_${VOICE}: sdltts.o $(flite_LIBS_deps)
$(TOP)/tools/make_voice_list $(VOICE)
rm -f flite_voice_list.o
$(MAKE) flite_voice_list.o
-       $(CC) $(CFLAGS) -o $@ flite_main.o flite_voice_list.o $(flite_LIBS_flags) $(LDFLAGS)
+       $(CC) $(CFLAGS) -o $@ sdltts.o flite_voice_list.o $(flite_LIBS_flags) $(LDFLAGS)

install:
#       The basic binaries

tools/Makefile.flite diff:

Use this diff too if you want to use only one voice at a time. It outputs smaller binaries next to sdltts.

diff -ur flite-1.4-release/tools/Makefile.flite flite-1.4-webos/tools/Makefile.flite
--- flite-1.4-release/tools/Makefile.flite      2009-08-22 11:24:37.000000000 -0700
+++ flite-1.4-webos//tools/Makefile.flite       2011-10-09 04:29:37.264633493 -0700
@@ -102,8 +102,8 @@
$(VOICENAME)_mcep.o: $(VOICENAME)_mcep.c
$(CC) -I. -I$(FLITEDIR)/include -c -o $@ $<

-flite_$(VOICENAME): flite_main.o flite_voice_list.o $(FLITELIBS) lib$(VOICENAME).a
-       $(CC) $(CFLAGS) -o $@ flite_main.o flite_voice_list.o $(LOCAL_LIBS)  $(LOCAL_LANGLEX_LIBS) $(FLITELIBFLAGS) $(LDFLAGS)
+flite_$(VOICENAME): sdltts.o flite_voice_list.o $(FLITELIBS) lib$(VOICENAME).a
+       $(CC) $(CFLAGS) -o $@ sdltts.o flite_voice_list.o $(LOCAL_LIBS)  $(LOCAL_LANGLEX_LIBS) $(FLITELIBFLAGS) $(LDFLAGS)

.build_lib: $(OBJS)
@ $(AR) cruv $(LIBDIR)/lib$(VOICENAME).a $(OBJS)

new main/sdltts.c file:

Update 10/12/11: It might be late for most of you, but unwiredben just sent an updated sdltts.c. It runs the audio in a separate thread so your javascript can continue unabated. Thanks Ben!

His version (it doesn’t compile) is at https://gist.github.com/1281621. The one below compiles and is available down at the bottom.

/*************************************************************************/
/*                                                                       */
/*                  Language Technologies Institute                      */
/*                     Carnegie Mellon University                        */
/*                         Copyright (c) 2001                            */
/*                        All Rights Reserved.                           */
/*                                                                       */
/*                           Fahrzin Hemmati                             */
/*                         Copyright (c) 2011                            */
/*                                                                       */
/*  Permission is hereby granted, free of charge, to use and distribute  */
/*  this software and its documentation without restriction, including   */
/*  without limitation the rights to use, copy, modify, merge, publish,  */
/*  distribute, sublicense, and/or sell copies of this work, and to      */
/*  permit persons to whom this work is furnished to do so, subject to   */
/*  the following conditions:                                            */
/*   1. The code must retain the above copyright notice, this list of    */
/*      conditions and the following disclaimer.                         */
/*   2. Any modifications must be clearly marked as such.                */
/*   3. Original authors' names are not deleted.                         */
/*   4. The authors' names are not used to endorse or promote products   */
/*      derived from this software without specific prior written        */
/*      permission.                                                      */
/*   5. Modifications must be sent back to the author at their provided  */
/*      email address.                                                   */
/*                                                                       */
/*  CARNEGIE MELLON UNIVERSITY AND THE CONTRIBUTORS TO THIS WORK         */
/*  DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING      */
/*  ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO EVENT   */
/*  SHALL CARNEGIE MELLON UNIVERSITY NOR THE CONTRIBUTORS BE LIABLE      */
/*  FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES    */
/*  WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN   */
/*  AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,          */
/*  ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF       */
/*  THIS SOFTWARE.                                                       */
/*                                                                       */
/*************************************************************************/
/*             Author:  Fahrzin Hemmati (fahhem@fahhem.com)              */
/*               Date:  October 2011                                     */
/*************************************************************************/
/*                                                                       */
/*  Flite as a Hybrid PDK plugin                                         */
/*                                                                       */
/*************************************************************************/

#include "flite.h"
#include "flite_version.h"

#include "SDL.h"
#include "SDL_mixer.h"
#include "PDL.h"

extern cst_val *flite_set_voice_list(void);

void playText(const char * text) {
    cst_wave * w;
    w = flite_text_to_wave(text, flite_voice_select(NULL));

    if(Mix_OpenAudio(w->sample_rate, AUDIO_S16SYS, w->num_channels, 4096))
        exit(1);

    Mix_Chunk * sound = Mix_QuickLoad_RAW((Uint8*)w->samples,w->num_samples*sizeof(short));
    if(sound == NULL)
        exit(2);

    int channel = Mix_PlayChannel(-1, sound, 0);
    if(channel == -1)
        exit(3);
    while(Mix_Playing(channel) != 0);

    Mix_FreeChunk(sound);
    Mix_CloseAudio();
}

PDL_bool playAudio(PDL_JSParameters * parms) {
    const char * text = PDL_GetJSParamString(parms,0);

    /* since we don't process this in the method thread, instead post a
    * SDL event that will be received in the main thread and used to
    * launch the code. */
    SDL_Event event;
    event.user.type = SDL_USEREVENT;
    event.user.code = 0;
    event.user.data1 = strdup(text);
    SDL_PushEvent(&event);

    PDL_JSReply(parms, "{}");

    return 0;
}

void TTS_Init() {
    flite_init();
    flite_voice_list = flite_set_voice_list();
}

int main(int argc, char ** argv) {
    if(argc==2)
    {
        // don't do SDL loop
        flite_init();
        flite_voice_list = flite_set_voice_list();
        playText(argv[1]);
        exit(0);
        return 0;
    }

    // Initializes the video subsystem
    if (SDL_Init(SDL_INIT_AUDIO | SDL_INIT_VIDEO) < 0) {
        exit(1);
    }
    PDL_Init(0);
    TTS_Init();
    atexit(SDL_Quit);
    atexit(PDL_Quit);

    PDL_RegisterJSHandler("playAudio",playAudio);
    PDL_JSRegistrationComplete();
    PDL_CallJS("ready", NULL, 0);

    SDL_Event Event;

    while (1) {
        SDL_WaitEvent(&Event);
        if(Event.type == SDL_USEREVENT && Event.user.code == 0) {
            playText((const char *)Event.user.data1);
            PDL_CallJS("playbackComplete", NULL, 0);
            free(Event.user.data1);
        }
        if(Event.type == SDL_QUIT) exit(0);
    }
}

new make_for_dev.sh script:

export CFLAGS="-I/opt/PalmPDK/include/ -I/opt/PalmPDK/include/SDL/"
export LDFLAGS="-L/opt/PalmPDK/device/lib/ -lSDL -lSDL_mixer -lpdl -Wl,--allow-shlib-undefined"
export PATH=$PATH:/opt/PalmPDK/arm-gcc/bin
if [ x$1 != xclean ]; then
    ./configure --host=arm-none-linux-gnueabi
fi
make $*

new make_for_emu.sh script:

export CFLAGS="-I/opt/PalmPDK/include -I/opt/PalmPDK/include/SDL"
export LDFLAGS="-L/opt/PalmPDK/emulator/lib/ -lSDL -lSDL_mixer -lpdl -Wl,--allow-shlib-undefined"
export PATH=$PATH:/home/fahhem/projects/widk/sourcery/ia32-2010.09/bin/
if [ x$1 != xclean ]; then
    ./configure --host=i686-pc-linux-gnu
fi
make $*

Move the ttstest_plugin_appinfo.json to sdltts_plugin_appinfo.json inside the given app/ because that was misnamed.

Compiling and running

With those files in place, just run make_for_dev.sh or make_for_emu.sh and you’ll have your new binary as bin/sdltts. If you want to use only one voice at a time, you can use any of the bin/flite_$(VOICE) binaries instead, and they’re smaller as a result. Now run this script after compiling in order to send it to your device/emulator:

cp ../flite-1.4-webos/bin/sdltts app/sdltts # or copy bin/flite_cmu_us_kal to match the binary from HP
cp -r app app-pkg
# remove provided sdl app
rm app-pkg/arm_sdltts
# palm-package
palm-package app-pkg
# install ipk
palm-install com.joshondesign.webos.ttstest_1.0.0_all.ipk
rm -r app-pkg

That’s it. It’ll install it to whatever palm-install sends to.

On the device, you can just open the app and click the button and everything works fine. You can also run it via novaterm or ssh-ing in (if you ran pdk-device-install) by sending the text over the command line, triggering the if(argc==2) branch of main(), bypassing the SDL loop.

/media/cryptofs/apps/usr/palm/applications/com.joshondesign.webos.ttstest/sdltts "text to say"

The emulator is basically the same as the device, except you have to chmod it.

chmod +x /media/cryptofs/apps/usr/palm/applications/com.joshondesign.webos.ttstest/sdltts
/media/cryptofs/apps/usr/palm/applications/com.joshondesign.webos.ttstest/sdltts "text to say"

Conclusion

Now you have the source to your own binary and can build any extensions that you want. My only request is, according to part 5 of the license, is that you send me your changes to the code (I’ll post it online unless you request otherwise). I didn’t do this to win the contest since I already have a Touchpad, so the least you could do if you use my code to enter the contest (or for your own app in general) is to share back.

Update 10/9/11 11:48AM: I got it working on the emulator. Here are binaries for the single voice that HP gives for both the Touchpad and the emulator. You can use the instructions above to get those exact binaries (they will be at bin/flite_cmu_us_kal) or just download those to use.

Binaries:

You can download any of the following voices, choose the right platform and rename it to sdltts (or change your code):

Voice MD5 Touchpad MD5 Emulator
cmu_us_kal* Touchpad Emulator f94548eb870e474e5cb435043fbef9bc 35dd88db5f670674274257d04785472e
cmu_time_awb Touchpad Emulator 563409f4f00805fc17acc88ab399445c baef75b0d8f1524a7bc8ee5fee05d622
cmu_us_awb Touchpad Emulator 853b66b94dcf5e3beb265f6559034872 76cb6e33daaebfd6e51a880a710f4ed9
cmu_us_kal16 Touchpad Emulator 1885f3383126bffaa9d767e2f3ed2186 38ec555a367c00c80645a87f4d614a43
cmu_us_rms Touchpad Emulator 0c9538b69f0daa00970e48986f3112de 88b812f0bc941d0347721759d1d55024
cmu_us_slt Touchpad Emulator 4c85a417f3b81ad151431b12a5e8876d 92804c22df06922d3d266dd60d7ff56f

* HP gave everyone cmu_us_kal, so you can use any of the others if you want a little more distinction

Update 10/12/11: These binaries now play the audio in a background thread and call “playbackComplete” when finished.

10 thoughts on “WebOS’s Weekly App Hack 10-05-11

  1. You have done a hard work to customize the sdltts binary. But I am not familiar with linux command line and my conclusion is : the Weekly App Hack seems dedicated to low-level software developers and not to explore HMI and POO concepts.
    With the starting example, I was believing that will open a key to hybrid app just to add low-level functions reusable to a new kind of application.
    So, I haven’t a TouchPad yet and without a sdltts binary working for the emulator, I am not able to push the nice idea of ‘developing a reader/teacher for child or for foreign languages’… Thanks for your work.

    Reply

  2. Thx for your example. I was able to compile it under Windows in a cygwin environment. I had to replace some spaces in your Makefile diff into tabs to please make. Also the correct include path gave me a headache. Now it compiles. 2 strange things: the executable (probably stripped by the -s switch) is 16MB versus 3MB in the example. Also the sound starts ok but gets scattered at the end. this wasn#t the case in the original file. Do you know the reason?

    Reply

    Fahhem Reply:

    No, and I hadn’t looked into it before either. It’s probably too many libraries being statically linked in. The reason is that it’s the full binary with all the voices included. They gave only the one voice: cmu_us_kal. I’ll update the post for that.

    Reply

  3. Pingback: HP webOS Developer Blog

  4. Following the steps in the example from HP, I ended up with an app that has TTS, however, it only works until something, anything wants to play another sound. As soon as the system plays a sound, the plugin drops dead. At least, that’s the way it seems here.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>