11 December 2016

How I Found the Needle in the Haystack of WSO2 ESB Code


Modern-day business companies have many software products to optimize internal processes. The main goals are automating complex and repetitive tasks, fast reacting on problems and saving time of employees. Often these products work independently without any communication between each other and they are satisfying in narrow field of aspects.

Furthermore, any kinds of needed synchronization leads to manual activities to reproduce the same steps on other systems, so it may even reduce productivity of employees. Solution for that is to integrate systems together. On IT market we can find some middlewares that meet the integration requirements, like Tibco, webMethods, Oracle ESB, WSO2 ESB. The last of those is fully open source software and I had a pleasure to test it in real world.

What we need and what we have

Our business requirement was to integrate FrontEnd application with BackEnd application, implemented in AngularJS and Java, respectively. Choice fell on WSO2 ESB, since it’s free, scalable, lightweight software with plenty of functionalities. We decided to use latest version, WSO2 ESB 5.0.0, especially because of its new feature we wanted to use as communication protocol – WebSockets.

WSO2 ESB is very complex product and getting to know it takes a little more time than reading this humble article. Nevertheless, it’s worth to familiarize with the general rule of this approach, so let’s take a quick look on the chart below, which describes the architecture and integration between FrontEnd application and WSO2 ESB components.

Shortly about WebSocket

Now let’s see how WSO2 simplifies the creation of WebSocket communication tunnel to transfer the data. To use WebSocket connection I had to create Websocket inbound EndPoint in WSO2 project.

After that, I just had to send messages over defined WebSocket transport layer. It’s worth noting that I was sending data per dedicated user’s session channel – sessionId.

With such defined functionalities I could deploy the project on WSO2 ESB server and feel lucky that in few steps I defined communication channel without any piece of code. Hurray!

The needle

Wait a moment… Why after few days of working properly, an error of reaching maximum connections had appeared? WSO2 server was suspended and hasn’t been processing any data. Logs showed me an error: No buffer space available (maximum connections reached?)
I was googling the problem and redefining WebSocket endpoints in hundreds of different ways for two days in a row. After that time, I knew nothing, but the fact, that during each sending to WebSocket channel, WSO2 creates around 16 TCP/UDP ports without closing them. I decided to raise a hand and report a bug on WSO2 JIRA system – ESBJAVA-4915.

In the meantime, I started deeper investigation on my own. From the WSO2 ESB logs I could see that whole WebSockets mechanism is implemented inside of component called org.wso2.carbon.inbound.endpoint. The only thing to do was to download it and build locally. I started debugging line by line to understand what was going on and I found out that the following piece of code was responsible for caching WebSocket channel.

Generally, the parameter called clientIdentifier is being used for caching the user’s session channel and for keeping away from constant creations of a new channel for each WebSocket request. In practice, clientIdentifier stores every WebSockets channel topic name.

So why it is not working?

If you look carefully again on code above, you can see that if handlerMap contains any data, so if there is at least one channel created, the connection channel for clientIdentifier is not added to the handlerMap. It means that only the first connection is cached and every next connection is lost by cache mechanism.


The solution was very easy. All I needed to do was to correctly update hanlderMap in „else” statement.

After testing this hot-fix, I created and described the issue on Github („Redundant open ports – handlerMap is not updated after initialization”). I also proposed fix that works and in next few hours I was asked to create a pull request to master branch. At the moment, pull request 754 is accepted and this code fix will be included in next release of WSO2 ESB – WSO2 ESB 5.0.1.

Conclusion is pretty simple and fair,
Problem may be in the computer, not always in chair.

Adam Zaręba
Software Developer

Java developer. Passionate about all kinds of backend technologies and EAI middlewares. Co-founder of Java User Group Białystok. Fan of Dragon Ball and Sicilian mafia movies