Wolfmans Howlings

Using Postgresql with Grails

Jim Morris — Wed, 11 Nov 2009 14:11:41 -0800

I started playing with Grails, and I am planning to rewrite the blog engine I use for this blog in Grails. As noted earlier I wrote it in Merb originally. Although Merb is nice and lite it seems to be having difficulty keeping up with the gems it is dependent on. A case in point is Cucumber which it says is its recommended way of doing integration tests. The current version of Cucumber is very difficult to get working with Merb. See the Git source to see how I finally did it.

While I was learning Grails, I hooked up Postgresql 8.2 to Grails using the Datasource.groovy file. I turned on SQL logging, and to my horror saw how it was handling the id's. I have been using Postgresql for quite a while now, and used to run into problems with the creation and retrieval of the automatically generated IDs. It seems a lot of people run into these hard to describe problems, as it appears to happen rarely and is probably a race condition. With version >= 8.2 Postgresql introduced a way of inserting a new record and returning the automatically generated id in one atomic statement. This solves all the problems and is more efficient as it does not require two queries for each insert.

insert into users (version, crypted_password, admin, name, date_created, salt)
        values (?, ?, ?, ?, ?, ?) RETURNING id

For id when it has this schema

id | bigint | not null default nextval('users_id_seq'::regclass)

The RETURNING id is the trick here.

By default Hibernate, which is the underlying ORM used by GORM, which is the Grails ORM, seems to use a sequence to generate the next id then write the id in the INSERT SQL statement.

select nextval ('hibernate_sequence')
        
        insert into users (version, crypted_password, admin, name, date_created, salt, id)
        values (?, ?, ?, ?, ?, ?, ?)

This sequence should work in most cases, as nextval is supposed to be atomic and should return a unique sequence regardless of how many threads are concurrently accessing the database, however I have read that this causes problems in Hibernate and in Grails a bug was filed saying that this causes problems in Grails if you are doing inserts during the bootstrap process, something to do with the way Hibernate caches or batches stuff.

If you force hibernate to use the identity method of sequences id generator:'identity' which is the preferred method anyway, then Hibernate uses this sequence to get the id after an insert...

insert into users (version, crypted_password, admin, name, date_created, salt)
        values (?, ?, ?, ?, ?, ?)
        
        select currval('users_id_seq')

Now I know that this sequence (other than being inefficient) can be subject to a race condition where the id returned is not the id that was just generated. I do not know why as Postgresql says it should work, but I have actually been bitten by that on occasion in production, and it was solved by using RETURNING id.

So I was surprised that the current version of Hibernate does not use the currently recommended way of doing this common sequence. However it does provide a way to work around the problem. This solution really only works in Grails, because you have to hard code the column used for the id, if the API in Hibernate was a little more flexible it could be made generic, but that is an issue for the Hibernate JIRA.

Anyway here is the solution for Grails.

First stick this file in GRAILS_PROJECT/src/java/com/e4net/hibernate/dialect/PostgreSQL82Dialect.java

package com.e4net.hibernate.dialect;
        
        import org.hibernate.dialect.PostgreSQLDialect;
        
        public class PostgreSQL82Dialect extends PostgreSQLDialect {
        
            public PostgreSQL82Dialect() {
        
            }
        
            public boolean supportsInsertSelectIdentity() {
                    return true;
            }
        
            public String appendIdentitySelectToInsert(String insertString) {
                    return insertString + " RETURNING id";
            }
        }

This sets up a extension to the standard Postgresql dialect that Hibernate uses. It tells Hibernate that the id can be returned in the same SQL statement of the INSERT, and also how to modify the insert SQL statement to do that. This is where id is hard coded, if we were passed a little more information like what the identity column was we could make this generic.

Then in your GRAILS_PROJECT/grails-app/conf/DataSource.groovy

            dataSource {
                            dbCreate = "update" // one of 'create', 'create-drop','update'
                            url = "jdbc:postgresql://localhost/wolfmanblog_dev"
                            driverClassName = "org.postgresql.Driver"
                            dialect = 'com.e4net.hibernate.dialect.PostgreSQL82Dialect'
                            logSql = true
                            username = "xxxx"
                            password = "xxxx"
                    }

Note the dialect = 'com.e4net.hibernate.dialect.PostgreSQL82Dialect' this tells it to use the class we created above.

Now we need to tell Hibernate to always use the identity method for generating ids. This can be done in the static mapping stanza in each of your domain classes...

static mapping = {
            id generator: 'identity'
        }

Or if you are using Grails 1.2-M4 or better you can put this in GRAILS_PROJECT/grails-app/conf/Config.groovy

grails.gorm.default.mapping = {
            id generator:'identity'
        }

Which will change the default for all your domain classes.

Now when you do an insert you will get this...

insert into users (version, crypted_password, admin, name, date_created, salt)
        values (?, ?, ?, ?, ?, ?) RETURNING id

Which is more efficient and correct.

I'd like to see this in the default Hibernate Postgresql dialect, maybe it could determine the version of Postgresql it is using and do the correct thing?

Converting a Sidewinder 3D pro joystick to USB

Jim Morris — Sat, 24 Oct 2009 20:58:07 -0700

In playing with my new Rovio I decided that my Old MS Sidewinder 3D Pro Joystick would be an excellent way to control it, as it has the twist which can rotate the Rovio, and the joystick up/left/down/right can move the Rovio in those directions while still facing forward.

The problem is that the joystick has a game port connector, and my Linux workstation does not have a game-port. After doing the obligatory Googling I found this Exactly what I needed, except this was for Windows not Linux. Thinking I would have to modify the code I contacted Grendel who graciously sent me the source code for the project, however it turns out the code he wrote was so good it works as is on Linux, I just needed to modprobe sidewinder and it worked.

The project is based on a great little hackers device called a Teensy a relatively cheap general purpose programmable USB dev board. It has lots of sample code showing it being a keyboard or mouse or serial port etc. It is programmed in C using the avr-gcc toolchain. This is like a Basic Stamp with USB, something I needed years ago but couldn't find. Now I have one I forgot what I needed it for many years ago, but it is in my arsenal now.

The only thing I needed to do, as I was on Ubuntu Hardy (8.04 LTS) and not 8.10 or newer, was to build the avr-gcc toolchain myself. The one included with Hardy did not recognize the atmel mmcu atmega32u4 type, which is relatively new. This process is made much easier with a script you can get from this thread, Note you need to register with the forum to see the download links. The file you want is build-avr-gcc-4.3.3-libc-1.6.7-insight6.8-arch25-fix.zip. Make sure you read the first post of the thread and make sure you have all the dependencies installed first according to the pre-reqs.txt file in the download. (Especially the libmpfr-dev which I missed initially).

Anyway I prototyped the circuit using the photos provided in Grendels post, and plugged it into a Windows XP system and it worked, then I plugged it into my Linux box and dmesg told me it recognized the joystick device..

[926661.299475] input,hidraw8: USB HID v1.11 Joystick [Detlef <Grendel> Mueller Microsoft SideWinder 3D Pro (USB)] on usb-0000:00:1d.7-4.3

I then downloaded jstest which is a simple joystick test program, and all the degrees of freedom were recognized.

I wanted to make the proto more robust so I designed a PCB layout using ExpressPCB's free PCB design and schematic tool (Windows based but runs fine under Wine on Linux). These guys will make 3 off 3.8" x 2.5" PCBs MiniBoard for around $50, which is a get price if you can't make your own. However I decided that for this one off project I'd just do a PCB prototype instead, so I sent the PCB layout to Grendel as thanks for all his hard work, in case he ever wants to go into production for this great little project.

I got a Schmartboard board (around $5 from Fry's) and a Rt Angle female pcb mount 15 pin DSub connector (.99c), and soldered the components and jumper wires. The DSUB needed coaxing onto the board as I guess the pin spacing is not the standard 0.01", however Shmartboard does have a board especially for dsub connectors, Fry's did not stock it though. I like these Schmartboards, they are cheap and well built, through hole, and you can even do surface mount if you have to. They are a lot better quality than what you can get from Radio shack!

As a side note, as I am getting older and it has been 10 years since I did any pcb soldering I found that my eyesight has deteriorated to the point that I needed a magnifying lamp to do this work, and that my hand is no longer as steady as it was, and that my old trusty Weller 35W pro soldering iron was too hot and unwieldy, and I was making solder bridges everywhere and melting the wire and connectors. (That did not used to happen when I was younger ;) So I splashed out and got a nice new Hakko 936 soldering station, with the 907 iron and a very fine soldering tip and some very fine solder. This made this go a lot smoother, although finding the right temperature to use was hit and miss.

The last component was putting the finished product into a box. None of the Radioshack boxes I had lying around would fit, so remembering that I had great fun building an Acrylic box for my PC (which I eventually ditched for a Silenced case), I made a little acrylic box for the converter. A trip to Tap Plastics to get the adhesive was all that was needed as I still had some Acrylic scraps laying around.

Anyway that was my project for this week. Next up writing the code to control the Rovio with my Sidewinder 3D Pro. What Language to use?? Ruby, C++/Qt, Java, maybe learn a new one like Groovy? Hmmm what a Dilemma.

Fun with a PC104 board, embedded Linux and wifi

Jim Morris — Wed, 21 Oct 2009 01:37:39 -0700

Around 10 years ago I was playing with some home robotics, built a simple robot, with some sensors and an on-board Linux-based PC. The purpose was to experiment with Robot AI, a continuation of my PhD thesis I started on some 30 years ago, but did not complete.

The robot had a camera, a digital compass, a sonar scanner and a short range IR range detector, plus some bump detectors. I could control it over a wifi connection, that was a Orinoco PCMCIA board plugged into the PC104, 586 based Linux PC that was on-board. All powered by several batteries.

I ran out of time and interest and energy and shelved the project. 10 Years later I stumbled upon a relatively cheap camera, wifi based robot called a Rovio made by Wowwee, its basically a toy, but a cool one. The Rovio has much better manouverability than my feeble attempt, and it has a camera and is controlled over Wifi by some well documented HTTP calls.

I bought one of these gizmos to play with, and to "use" as a remote security camera to check on the dogs when I am not home. It fired up my Robot hobby enthusiasm, and as I have a lot of time on my hands these days I dug out all my old stuff, which I had mostly forgotten, and dusted off the old PC104 embedded system and tried to get it up and running again. I'm not sure what I'll use it for but it was pretty expensive back in the day, so I'll find some use for it. Maybe I'll stick it on the Rovio to give it some more computing power.

Anyway things have changed a lot since then, a PC104 is ISA based, and this board had no USB ports, but I did have a PCMCIA slot, which was 5v 16 bit only. This posed a problem as all PCMCIA cards today are Cardbus 3.3v and usually 32 bit. The only wifi card I had that was 5v was the old Lucent Orinoco 802.11b card. But my AP these days is a WPA/WPA2 based 802.11b/g.

The first fun thing was to get wpa_supplicant running on this old thing, if possible. I mentioned no USB because one workaround was to use a newer wifi card, I also wanted to use some flash drives. But this turns out to be impossible as no ISA/PC104 USB cards were ever built (according to an extensive Google search).

The board has 32Mb of RAM, and that was fully loaded, it also ran WhiteDwarf Linux out of a 32Mb disk-on-a-chip board that plugs into the IDE socket. The old version of WDLINUX I had definitely would not run the newer Hermes Linux drivers and wpa_supplicant. I found that Whitedwarf is still in business and have a newer version of wdlinux2.2. This is a Slackware based Dist, that fits in under 32Mb of Ram and Disk. But I want to run Ruby on this thing, and probably Java, so I needed some more room which meant swap space, and swap and flash drives don't go well together, so I dug out a 4Gb Seagate Microdrive I bought at Frys a few years back, which would do really nicely. However getting it to work reliably turned out to be tricky. I grabbed the latest version of wdlinux, and did some digging to find it seems to be based on Slackware 8.1, which is important as you need more packages to make it useful than wdlinux has, so installing from Slackware makes things easier. They have an ISO so I installed the ISO in a Vmware Workstation, so I could compile a new kernel and compile any application on my workstation, and then tried to install it also on the 4Gb microdrive. This worked eventually but it turns out the microdrive likes to go to sleep every few seconds, (I guess to save power when it is plugged into a camera or whatnot). This caused Linux to get upset (the kernel version is 2.4.29). I saw a lot of errors saying "no drq after write", this was caused by it going to sleep, as I found if I did

> dd if=/dev/hda of=/dev/null`

in a separate terminal, it kept it awake. The long term workaround was to execute

> hdparm -B255 /dev/hda

as soon as possible after boot, this turns off the power saving mode. I put it in /etc/rc.d/rc.S which is the first thing to get executed on startup. This error is more than annoying it corrupted the disk more than once.

I ordered a compact flash to ide44 adapter and short 44pin f-f cable so I could plug the microdrive into the ide connector on the board.

So now I had wdlinux 2.2 installed on a 4Gb microdrive on the PC104 board. Luckily 2.2 came with PCMCIA and associated tools already built into the kernel, unfortunately the module for my particular PCMCIA adapter board was not built, so I built a new kernel with the relevant driver built as a module, and was able to get the PCMCIA system up, plugging in the Orinoco card loaded the orinoco_cs driver which would work fine if I was still using WEP. After much Googling I found that the Orinoco which is a Hermes-I based card, would apparently work with wpa_supplicant if you upgraded the firmware and used a Linux driver released by Agere (who have since been acquired and their web site dismantled) Luckily I found the source online somewhere (Google for wl_lkm_718_release.tar.gz), and built it. The driver is called wlags49 and comes in a Hermes-I and Hermes-II flavors and for pccarc and pci, also the firmware is downloaded and can be either station or ap modes. After browsing through the source code they provided (which BTW only works on 2.4.x kernels) I figured out how to build it properly and also build the version of wpa_supplicant and Hermes driver that came with it. (I saw that some people have provided patches to make the code compile under 2.6.x kernels, but that didn't help me too much).

Initially the driver didn't seem to work too well, iwlist eth1 scanning did not seem to return anything, and wpa_supplicant also barfed when it tried to scan for AP's. A bit more digging and reading the sparse docs, looked like I needed to do some configuration of the module when it was loaded, rather than allow wpa_supplicant to do all the configuration. They provide a way to configure the Agere based chips with a /etc/agere/iwconfig-eth1 file. I set the SSID, the download_firmware and the debugging flags, and it seemed to fix everything. The wpa_supplicant.conf file was pretty sparse as it was a really old version of wpa_supplicant they were using, but it was enough to finally get a connection via WPA to my AP. Note WPA2 is not supported.

Along the way I was looking for alternatives, like using a newer PCMCIA card, I had a 3.3v newer card, but that wouldn't plug in to the 5v adapter, and I couldn't find a 5v to 3.3 v adapter on the market, which is odd as it is one chip to do it. I also couldn't find many 5v wireless cards, although it looked like I could still buy a Linksys WPC11 card so long as it was v1, v2 or v3, the new ones are v4 and appear to be 3.3v only.

So things learned...

There are no USB boards for ISA (or PC104) based PCs
New PCMCIA cards are 3.3v and won't work on older PCMCIA adapters (or Laptops)
There are no converters that allow you to run a newer 3.3v PCMCIA card in a 5v slot
Orinoco cards can be made to work with WPA, but require newer firmware and the Agere drivers.
working with old H/W is a pain, but embedded boards such as PC104 are still using old technology, although PC104+ does have a PCI bus which allows newer boards to be used
Using a compact card microdrive to run Linux off of is possible but you need to disable the power management in the drive

Now I finally have my old PC104 board running again with its wireless card and talking WPA, I have to figure out what to do with it :)

My next project is to build a USB adapter for my old Microsoft Sidewinder 3D Pro joystick, so I can use it on Linux to control my Rovio. This has already been mostly done but for Windows. I am going to modify it to talk a serial protocol so I can plug it into my Linux workstation.

A TCP socket bridge for Javascript

Jim Morris — Sun, 13 Sep 2009 22:40:03 -0700

In the process of rewriting my voice server in Erlang, I decided a new web-based Javascript UI would also be welcome. One problem of course is my voice server requires a TCP connection and a UDP socket for sending voice, not to mention the whole voice capture, playback thing.

In order for this to work the voice and playback would need to be written in a Java Applet, so it made sense to put the whole TCP/UDP communications stuff in there too.

Firefox uses Liveconnect in its Javascript that allows Javascript to call methods in a Java Applet directly, and also for Java to call Javascript functions, as Javascript has no way of opening an arbitrary TCP socket that would have to be in the Java Applet.

One thing that seems to not be documented in the Livescript docs, is that the Java method that is called directly from Javascript is not run with any privileges at all, including the one that allows TCP connections back to the same server that served up the Applet, or any privileges that a signed Jar would have. So if you try to call an open socket from javascript you will get the dreaded error: java.security.AccessControlException: access denied (java.net.SocketPermission 127.0.0.1:3000 connect,resolve)

I found by Googling that a workaround is to run a thread from the Java Applet start() method, and pass any commands that require privileges to that thread. This thread must be started from the Applet.start() method, not from any method called from Javascript. However this will not work if the page is opened as a file, only if it is served over http.

I came across many clever methods of doing this in my research, one was to pass a Runnable into this thread, so you can have that thread run arbitrary Java code. However I wanted something a little simpler as for now I would do a TCP socket bridge, which only required three calls, Open, Close and Write. There was no need to pass in a Runnable to do that. The Open needed to be a synchronous call, the close and write could be asynchronous.

The relatively simple solution I came up with using the concurrency libraries in recent versions of Java is shown here, other versions can be found on the web.

After starting up the Communication Thread from the Applet start(), I pass in one of the commands I want it to execute, I use the SynchonousQueue class, to make sure only one command can be executed at a time, and in the case of open() I use another SynchonousQueue to return the results. This makes the open() call totally synchronous and will block (and hang the browser) until either the socket is opened, fails to open, or a timeout occurs.

I am writing JSON strings to the server as that provides a nice way to do a simple RPC protocol that can be easily generated in Javascript, and I return JSON as again Javascript can handle that easily. As the server is Erlang I use {packet, 2} in the TCP socket meaning each packet is preceded by the BigEndian count of the size of the packet, this gets over the need to make sure the entire JSON has been received before decoding it.

If you do not want to sign your applet in a jar then you need to only connect to the same server the applet is served from, so the code allows for open to only specify the port, and it figures out the host name. There is also an open call that specifies an arbitrary host, but for this to work you need to sign the jar. (In my case I will need to sign the jar eventually as the capture audio calls require special privileges). There are plenty of howtos on the web about self-signing jars so I won't go into it here.

Onto the code...

package com.e4net.socketbridge;
        
        import java.applet.Applet;
        import java.io.BufferedOutputStream;
        import java.io.InputStream;
        import java.io.OutputStream;
        import java.net.Socket;
        import java.net.URL;
        import java.nio.ByteBuffer;
        import java.nio.ByteOrder;
        import java.util.concurrent.SynchronousQueue;
        import java.util.concurrent.TimeUnit;
        
        import netscape.javascript.JSObject;
        
        /**
         * @author Jim Morris, e4 Networks LLC
         * @license public domain
         */
        public class E4SocketBridge extends Applet
        {
            private static final long serialVersionUID = 1L;
        
            private JSObject win;
        
            private SynchronousQueue<Work> workQueue= new SynchronousQueue<Work>();
            private SynchronousQueue<Boolean> resultQueue= new SynchronousQueue<Boolean>();
        
            public enum WorkType{OPEN, CLOSE, WRITE};
        
            public void init() {
                win = JSObject.getWindow(this);
            }
        
            // an item of work passed into the comm thread
            protected class Work {
                private WorkType command;
                private Object data;
                private Work(WorkType command, Object data) {
                    this.command = command;
                    this.data = data;
                }
                public WorkType getCommand() {
                    return command;
                }
                public Object getData() {
                    return data;
                }
            }
        
            // Thread that reads incomign data from server and sens it asynchronously to Javascript
            private class ReadThread implements Runnable {
                private JSObject win;
                private InputStream ins;
                byte [] cntbuf= new byte[2];
                byte [] buf= new byte[65536];
        
                public ReadThread(JSObject win, InputStream ins) {
                    this.win= win;
                    this.ins= ins;
                }
        
                // reads all the bytes specified into the given buffer
                private boolean getBytes(byte [] data, int len){
                    int cnt= 0;
                    int t;
                    while(len > cnt){
                        try {
                            t= ins.read(data, cnt, len-cnt);
                        } catch (Exception ex) {
                            return false;
                        }
        
                        if(t < 0){
                            return false;
                        }
                        cnt += t;
                    }
                    return true;
                }
        
                public void run() {
                    try {
                        // read input and notify javascript
                        // all packets are prepended with big-endian 2 byte packet length, read the whole thing before sending a string to JS 
                        boolean running= true;
        
                        while(running){
                            try {
                                if(!getBytes(cntbuf, 2)){ // read in header count
                                    running= false;                         
                                    break;
                                }
                                int count= be_short(cntbuf); // bigendian 2 byte count
                                System.out.println("got count: " + count);    
                                if(!getBytes(buf, count)){
                                    running= false;                         
                                    break;
                                }
                                Object [] arg= new Object[1];
                                arg[0]= new String(buf, 0, count);
                                win.call("got_data", arg);
        
                            } catch (Exception e) {
                                running= false;
                            }
                        }
        
                        Object[] arg= new Object[1];
                        arg[0]= "closed";
                        win.call("closed", arg);
                    } catch(Exception ex) { }
                }
        
                // converts two bytes into a bigendian unsigned short
                private int be_short(byte[] buf) {
                    int l1= (int)buf[0]&255;
                    int l2= (int)buf[1]&255;
                    return ((l1 << 8) + l2);
                }
            }
        
            private Thread commThread;
            private CommRun commRun;
        
            // open the socket in a thread so signed jar applet privs work
            private class CommRun implements Runnable {
                private String host;
                private int port;
                private JSObject win;
                private Socket sock;
                private InputStream ins;
                private OutputStream outs;
                private BufferedOutputStream bos;
                private Thread readThread;
                private ByteBuffer bb;
        
                public CommRun(JSObject w){
                    this.win= w;
                    readThread= null;
                    // default is to open to the host the applet was loaded from
                    URL urlServer = getCodeBase();
                    host= urlServer.getHost();
                }
        
                void setHost(String host){
                    this.host= host;
                }
        
                void setPort(int port){
                    this.port= port;
                }
        
                private void open(){
                    System.out.println("open " + host + ":" + port);    
        
                    boolean b;
                    try {
                        sock= new Socket(host, port);
                        ins = sock.getInputStream();
                        outs = sock.getOutputStream();
                        bos= new BufferedOutputStream(outs);
        
                        // Use a ByteBuffer to automatically convert shorts to bigendian
                        bb= ByteBuffer.allocate(65538); // worst case size
                        bb.order(ByteOrder.BIG_ENDIAN);
        
                        readThread= new Thread(new ReadThread (win, ins));
                        readThread.start();
                        b= true;
                    } catch(Exception ex) {
                        ex.printStackTrace();
                        b= false;
                    }
                    try {
                        resultQueue.offer(new Boolean(b), 10, TimeUnit.SECONDS);
                    } catch (InterruptedException e) {
                    }
                }
        
                /**
                 * Send data to socket
                 *  
                 * @param data to send, will be a single String
                 * @throws Exception 
                 * 
                 */
                private void send(String data) throws Exception {
                    bb.clear();
                    // prepend string with size of string
                    packString((String) data);
                    bb.flip();
                    bos.write(bb.array(), 0, bb.limit());   
                    bos.flush();
                }
        
                private void packString(String data) throws Exception {
                    byte [] buf= data.getBytes();
                    int len= buf.length;
                    if(len > 32767)
                        throw new Exception("string too long");
                    bb.putShort((short)len).put(buf);
                }
        
                private void close(){
                    System.out.println("close");
                    try { sock.close(); } catch(Exception e){}
                    if(readThread != null)
                        readThread.interrupt();
                }
        
                public void run() {
                    boolean running= true;
                    Work work= null;
        
                    while(running){
                        try {
                            work= workQueue.take();
                        } catch (InterruptedException e) {
                            running= false;
                            break;
                        }
        
                        switch (work.getCommand()) {
                        case OPEN:
                            open();
                            break;
        
                        case CLOSE:
                            close();
                            break;
        
                        case WRITE:
                            try {
                                send((String)work.getData());
                            } catch (Exception e) {
                                e.printStackTrace();
                            }
                            break;
        
                        default:
                            break;
                        }
                    }
                }
        
            }
        
            // This is called when the Applet starts
            public void start() {
                System.out.println("Applet started");
                // start the communications thread that will run with the privilege of a Java Applet
                commRun= new CommRun(win);
                commThread= new Thread(commRun);
                commThread.start();
            }
        
            // The following methods are called from Javascript
        
            /**
             * Open a TCP socket to host:port or just port
             * Complicated because liveConnect (or the javascript to java bridge), does not run
             * any directly called methods under the signed jar privilege, so we need to signal the thread
             * which does run privileged to do the open.
             * We wait for the open to succeed or fail
             * 
             * @param host IP or host name
             * @param port port
             */
            public boolean open(String host, int port) {
                commRun.setHost(host);
                return open(port);
            }
        
            /**
             * Just open back to Host that served this Applet
             * Wait for the open to finish and return the result to Javascript caller
             * @param port
             * @return true if open succeeded else false
             */
            public boolean open(int port) {
                commRun.setPort(port);
                boolean b;
                try {
                    b= workQueue.offer(new Work(WorkType.OPEN, null), 10, TimeUnit.SECONDS);
                    if(b){
                        Boolean r= resultQueue.poll(10, TimeUnit.SECONDS);
                        b= (r == null) ? false : r.booleanValue();
                    } else
                        System.out.println("Timed out waiting for open");
                } catch (InterruptedException e) {
                    return false;
                }
                return b;
            }
        
            /**
             * write the String to the socket
             * @param data
             * @return true on success false on timeout or failure
             */
            public boolean write(Object data) {
                boolean b;
                try {
                    b= workQueue.offer(new Work(WorkType.WRITE, data), 10, TimeUnit.SECONDS);
                    if(!b)
                        System.out.println("Timed out waiting for write");
                } catch (Exception e) {
                    return false;
                }
                return b;
            }
        
            /**
             * Close the socket
             */
            public void close() {
                try {
                    workQueue.offer(new Work(WorkType.CLOSE, null), 10, TimeUnit.SECONDS);
                } catch (Exception e) {
                }
            }
        
        }

The applet is loaded as the following html shows with a simple sample...

<html>
        <head>
         <title>applet html page</title>
         <script type="text/javascript" src="e4socketbridge.js"></script>
        </head>
        
        <body>
        
          Test opening a socket to localhost:3000
        
          <p>
          <applet name="socketBridge" archive="e4socketbridge.jar" code="com.e4net.socketbridge.E4SocketBridge" width=1 height=1 mayscript="true" scriptable="true"></applet>
          </p>
        
          <input type="button" value="Open" onclick="openSocket()" />
          <input type="button" value="Write string" onclick="writeSocket('hello there')" />
          <input type="button" value="Close" onclick="closeSocket()" />
        <p />
        
        </body>

The javascript library that talks to the Java Applet is in e4socketbridge.js

        // open a TCP socket back to the server this was loaded from at port 3000
        function openSocket()
        {
          document.socketBridge.open(3000);
        }
        
        function writeSocket(data)
        {
          document.socketBridge.write(data);
        }
        
        function closeSocket()
        {
          document.socketBridge.close();
        }
        
        // called asynchronously from Java when the socket recieves a complete packet
        function got_data(ln)
        {
          alert("Got data: " + ln);
        }
        
        // called asynchronously from Java when the socket closes
        function closed()
        {
          alert("socket closed");
        }

Any incoming data from the server needs to be sent to the Javascript, to handle that I have a readThread that listens on the TCP socket for any incoming data, reads the entire packet based on the 2 byte header, then calls a Javascript function which would handle that event. I'm not exactly sure how Javascript handles asynchronous calls from different Java threads, but it seems to work OK, and there is some mention of Javascript being able to handle this kind of thing.

Note I put timeouts on any method that could hang, because the entire browser hangs when those calls into the Java Applet hang, and debugging that was ugly, having to kill off the browser everytime.

I use a 2 byte header so no packet can be bigger than 65k, this helps avoid DOS (Denial of Service) attacks where someone could tie up a process in the server by specifying it was going to send a huge packet, this could also cause the server to run out of memory. The downside is I have to manually break up anything that needs to send more than 65k. If you are not concerned about the potential DOS use {packet, 4} instead, and change it to send a 4 byte integer instead.