Building WS4D-gSOAP on Linux

WS4D-gSOAP is a framework that assists development of web services on multiple platforms. These includes mobile phones, server, computers or embedded system.

WS4D Features

WS4D can run as a standalone application or based upon a server and has been used in many commercial products. Despite being the most prominent framework for embedded system SOAP services, WS4D seems to be lacking behind in version support and documentation. Namely, you cannot build the latest version of WS4D-gSOAP with the latest version of gSOAP, even though both frameworks’ last version were one or two years ago. Development doesn’t seem to be active, so does the community. Finding help is a big hassle!

So, this post outlines some problems i encountered while working with the framework and how to solve them, hoping that the next follower won’t have as much trouble starting in a new environment.

Terminology

It seems that the WS4D documentation assumes that you have a certain level of understanding on Unix-based build system and working across platform. You cannot find definition for several new terminologies used in the documentation, so the docs may be quite confusing and intimidating for new users. I find these words particularly weird:

  • In source build: The binaries will be generated inside the same directory as the source. This has the advantage of simplifying the make files as you don’t have to link and copy libraries over, but the directory may look extremely messy.
  • Out-of-source build: Separate source and binary directories. This is the configuration used by WS4D tutorial and code. The sample make file in the tutorial already to the hard work and creates the most common directories used in almost all WS4D projects.
  • Cross-compile (and compilation): This means instead of compiling only one binary to use on your development platform, it will compile another binary for another platform. What do you do with this second binary is up to you: you can push it to the device or run it in a simulator, but cmake’s job in this case is only generating the binary file for you.
  • Tool-chain: A set of compiler and linker for a specific device or platform. Cross-compiling uses two or more tool-chain to generate two or more executable.

Proper version

The documentation for WS4D is a bit disorganized, so it may not relevant for first-time readers that you must compile WS4D with specific versions of gSOAP, doing otherwise will lead to make-errors and build headaches. It is mentioned in Features page (which is not visible from the first page of documentation). I was reading only the installation instruction and tutorial when I started to build the system, so I missed this important fact! How should I know that Getting Started is not a good place to get started?

ws4d-gsoap 0.7.x ws4d-gsoap 0.8.x ws4d-gsoap trunk
 gSOAP 2.8.0 Not yet supported Not yet supported Not yet supported
 gSOAP 2.7.17 Not yet supported supported with patch supported with patch
 gSOAP 2.7.16 Not yet supported supported with patch supported with patch
 gSOAP 2.7.15 Not yet supported Not yet supported Not yet supported
 gSOAP 2.7.14 Not yet supported Not yet supported Not yet supported
 gSOAP 2.7.13 supported supported supported
 gSOAP 2.7.12 supported supported supported
 gSOAP 2.7.11 supported supported supported
 gSOAP 2.7.10 supported supported supported

So the latest version of the stack you can use is gSOAP 2.7.17 with WS4D-gSOAP 0.8. I don’t know why it is so complicated, maybe due to the fact the gSOAP 2.8 also implemented WS-Discovery (a feature overlap with WS4D), and the auther didn’t have enough time to adjust WS4D-gSOAP to the new changes.

Build steps and permissions

Supposed you have decent Linux knowledge, you can use this inside the directory where WS4D-gSOAP was extracted to

  1. ./configure
  2. ccmake .
  3. Edit path to gSOAP directory (the source you downloaded, do not ‘make install’ this)
  4. Press g to generate, there may by some warning, ignore them
  5. Press c to configure
  6. make (I don’t know why this step is needed, but if you misses this step, the next step will result in various errors. This step will result in errors)
  7. su -c ‘make’ – (You can’t use sudo, you have to use su with a trialing dash to load root user’s environment. Root permission is required to install binaries to their proper locations)

Web service for device (WS4D) and using make, cmake with Integrated Development Environments (IDEs) on Linux

WS4D-gSOAP is a framework to deploy web services on multiple environments (computers, embedded devices, phones…) without having to rewrite code. More introduction information can be found in this post. The downside for WS4D is that the build system is based on cmake, and it’s a bit complicated to use. I have since managed to successfully build the stack from source (there’s no binary distribution). But I ran into numerous problems during my time getting familiar with the stack. Namely following the tutorial.

Even for such a simple task of copying and pasting code from the tutorial (The Air Conditioner tutorial), I did not succeed. The client and device doesn’t seem to be able to communicate with each other. Even though the tutorial have been kind enough to include logging code (send, received and memory allocations), I’m still unable to figure out where the problem lies. And that’s when I think I’m forced to perform debugging on the project. That leads to a big problem: what am I supposed to use to debug this?

So I tried to install Code Blocks, but it doesn’t support cmake projects, it couldn’t import the CMakeList.txt. Then reading through several articles revealed Eclipse to be a pretty good gdb front-end, I installed it, and have successfully imported the project, but I was hit with several problems:

  • I couldn’t build the project, Eclipse’s project structure was make-based, not cmake-based, trying to ‘make all’ or ‘make clean’ with Eclipse obviously would generate errors
  • Editing  the code is an eyesore because Eclipse doesn’t seem to be able to recognize include and libraries directories even after I added them manually in project properties.
After several more hours of analyzing WS4D-gSOAP documentation (not really helpful, they are only related to build and run with command line), cmake (wiki pages after wiki pages with no structure between pages and in articles), Eclipse and Code Blocks’ documentation on make projects; I finally realized the right way to do this: cmake supports generating make-based projects for both Code Blocks and Eclipse, and it’s as simple as this:
  1. Go to the directory where you would normally run cmake to build your project from the command line
  2. Perform

    cmake -G"Eclipse CDT4 - Unix Makefiles" -D CMAKE_BUILD_TYPE=Debug ../certi_src
    cmake . -G "CodeBlocks - Unix Makefiles"
  3. Import

    The project directory
    ProjectName.cbp file
You can now work with the project normally within your preferred IDE

Recover encrypted home folder under Ubuntu / Kubuntu

The community documentation can be of great help, namely the following section

  1. If you use encrypted filenames (standard in Ubuntu >= 9.04) you have to do the following first:
    •  sudo ecryptfs-add-passphrase --fnek 
    •  Passphrase:  (Enter the mount passphrase you recorded when you setup the mount–this passphrase is different from your login passphrase.)
    • You should now get two lines looking like this:
    •  Inserted auth tok with sig [9986ad986f986af7] into the user session keyring 
    •  Inserted auth tok with sig [76a9f69af69a86fa] into the user session keyring  (write down the second value in the square brackets)
  2. Mount using sudo:
    •  sudo mkdir -p /home/username/Private  
    •  sudo mount -t ecryptfs /home/username/.Private /home/username/Private 
    •  Selection: 3  (use a passphrase key type)
    •  Passphrase:  (Enter the mount passphrase you recorded when you setup the mount–this passphrase is different from your login passphrase.)
    •  Selection: aes  (use the aes cipher)
    •  Selection: 16  (use a 16 byte key)
    •  Enable plaintext passthrough: n 
    •  Enable filename encryption: y  (This and the following options only apply if you are using filename encryption)
    •  Filename Encryption Key (FNEK) Signature:  (the value you wrote down from the second line above)

Problem is, like some folks at the Ubuntu forums, I encounted the following error when trying sudo mount -t encryptfs /source /destination:

Attempting to mount with the following options:
  ecryptfs_unlink_sigs
  ecryptfs_fnek_sig=xxx
  ecryptfs_key_bytes=16
  ecryptfs_cipher=aes
  ecryptfs_sig=xxx
Error mounting eCryptfs: [-2] No such file or directory
Check your system logs; visit

After fiddling around for a whole day, I finally figured out the problem: the blind confidence in Linux developers’ ability. You see, I used to believe Linux’s symlink always point to the same directory on the drive event after it was mounted somewhere else. So say if a link from your old drive, say ~/etc points to /etc, when you mount the root filesystem on another machine, it should now point to /media/mount/etc right? Wrong!

Problem is, the hidden .Private in the encrypted folder is only one such symlink, and when you attempted to mount it in another system, the symlink is broken. If you do a ls -l, you can see the symlink points to /home/.ecryptfs/<yourusername>/.Private relative to your old directory structure, so to successfully mount your encrypted folder, you shouldn’t mount /home/<yourusername>/.Private like mentioned in the tutorial, but instead

sudo mount -t ecryptfs /media/[old drive mount point]/home/.ecryptfs/[your old username]/.Private /home/[your new username]/[new mount point]

Problem solved!

KTouch lession maker

Some of you might have landed on this article searching for the book instead, here it is 🙂

I have been trying to improve my typing by learning Dvorak and so far I’ve been to be able to type more accurately (I’m not talking about speed here :p). I have been using the Grass Soft touch typing program to train on Windows, it is pretty useful except for the limited word list. They claimed that they have included 500 most used word in English. That just doesn’t seem enough to me!

Besides, I have also started to use Linux and I needed a program that will keep me busy on the new platform. Of course in the open source world there’s always a lot of option too choose from, most of them are free too. I have chosen KTouch as it seems to be under active development and also endorsed by KDE.

KTouch with my customized lesson

So far, it’s a great program! It automatically detects your keyboard layout and select the appropriate starting lesson. It have two lesson for Dvorak: ABCD and computer generated. Despite the name, the computer generated lesson does not change as you take it because it’s just a file pre-rendered by the developer. I hit the same obstacle I did with the previous touch typing program: no variety.

Fortunately KTouch allows you to load external lesson files (they are just XML files) and text file; but due to poor application design, the text file won’t let you type the whole file but only the first few sentences. Which means if you want to practice with a long text, you’ll have to split them manually into multiple files.

<?xml version="1.0" encoding="utf-8"?><KTouchLecture>
<Title>Puzo, Mario - The Godfather.txt</Title>
<Comment>This is a lession created from Puzo, Mario - The Godfather.txt</Comment>
<FontSuggestions>Courier 10 Pitch</FontSuggestions>
<Levels>
<Level>
<LevelComment>level 1</LevelComment>
<NewCharacters>a lot :)</NewCharacters>
<Line>Chapter 1</Line>
<Line>Amerigo Bonasera sat in New York Criminal Court Number 3 and waited for</Line>
<Line>justice; vengeance on the men who had so cruelly hurt his daughter, who had</Line>
<Line>tried to dishonor her.</Line>
<Line>The judge, a formidably heavy-featured man, rolled up the sleeves of his black</Line>
<Line>robe as if to physically chastise the two young men standing before the bench.</Line>
</Level>
</Levels></KTouchLecture>

Structure of XML lesson files

So I made a program with C# to make KTouch lesson out of text files. Thanks to C#’s ability to process strings and XML, this has been a fairly easy task, here’s the code:

            const int LinePerLevel = 6;
            const int CharactersPerLine = 80;
            FileInfo Info = new FileInfo(Filename);
            string[] RawData = File.ReadAllLines(Filename, Encoding.Default);
            XmlTextWriter Writer = new XmlTextWriter(Filename + ".xml", Encoding.UTF8);
            Writer.WriteStartDocument();
            Writer.WriteStartElement("KTouchLecture");
            Writer.WriteRaw("rn");
            {
                Writer.WriteElementString("Title", Info.Name);
                Writer.WriteRaw("rn");
                Writer.WriteElementString("Comment", "This is a lession created from " + Info.Name);
                Writer.WriteRaw("rn");
                Writer.WriteElementString("FontSuggestions", "Courier 10 Pitch");
                Writer.WriteRaw("rn");

                Writer.WriteStartElement("Levels");
                Writer.WriteRaw("rn");
                {
                    int LevelCount = 1;
                    int LineCount = 0;
                    string[] LevelLines = new string[LinePerLevel];
                    string Buffer = "";
                    for (int i = 0; i < RawData.Length; i++)
                    {
                        string TrimmedLine = RawData[i].Trim();
                        if (string.IsNullOrEmpty(TrimmedLine))
                            continue;

                        TrimmedLine = TrimmedLine.Replace('“', '"');
                        TrimmedLine = TrimmedLine.Replace('”', '"');
                        TrimmedLine = TrimmedLine.Replace('’', ''');
                        TrimmedLine = TrimmedLine.Replace('‘', ''');
                        TrimmedLine = TrimmedLine.Replace("t", "");

                        if (Buffer.Length > 0 && (Buffer[Buffer.Length - 1] == '.' || Buffer[Buffer.Length - 1] == '?' || Buffer[Buffer.Length - 1] == '!'))
                            Buffer = Buffer + " " + TrimmedLine;
                        else
                            Buffer = Buffer + TrimmedLine;

                        while (LineCount < LinePerLevel && Buffer.Length > CharactersPerLine)
                        {
                            int CutOffSpacePos = Buffer.LastIndexOf(" ", CharactersPerLine - 1);
                            LevelLines[LineCount++] = Buffer.Substring(0, CutOffSpacePos);
                            Buffer = Buffer.Substring(CutOffSpacePos + 1);
                        }

                        if (Buffer.Length < = CharactersPerLine && LineCount < LinePerLevel)
                        {
                            LevelLines[LineCount++] = Buffer;
                            Buffer = "";
                        }

                        if (LineCount >= LinePerLevel)
                        {
                            Writer.WriteStartElement("Level");
                            Writer.WriteRaw("rn");
                            {
                                Writer.WriteElementString("LevelComment", "level " + LevelCount.ToString());
                                Writer.WriteRaw("rn");
                                Writer.WriteElementString("NewCharacters", "a lot :)");
                                Writer.WriteRaw("rn");
                                for (int j = 0; j < LevelLines.Length; j++)
                                {
                                    Writer.WriteElementString("Line", LevelLines[j]);
                                    Writer.WriteRaw("rn");
                                }
                            }
                            Writer.WriteEndElement();
                            LevelCount++;
                            LineCount = 0;
                        }
                    }

                    while (!string.IsNullOrEmpty(Buffer))
                    {
                        while (LineCount < LinePerLevel && Buffer.Length > CharactersPerLine)
                        {
                            int CutOffSpacePos = Buffer.LastIndexOf(" ", CharactersPerLine - 1);
                            LevelLines[LineCount++] = Buffer.Substring(0, CutOffSpacePos);
                            Buffer = Buffer.Substring(CutOffSpacePos + 1);
                        }

                        if (Buffer.Length < = CharactersPerLine && LineCount < LinePerLevel)
                        {
                            LevelLines[LineCount++] = Buffer;
                            Buffer = "";
                        }

                        if (LineCount >= LinePerLevel)
                        {
                            Writer.WriteStartElement("Level");
                            Writer.WriteRaw("rn");
                            {
                                Writer.WriteElementString("LevelComment", "level " + LevelCount.ToString());
                                Writer.WriteRaw("rn");
                                Writer.WriteElementString("NewCharacters", "a lot :)");
                                Writer.WriteRaw("rn");
                                for (int j = 0; j < LevelLines.Length; j++)
                                {
                                    Writer.WriteElementString("Line", LevelLines[j]);
                                    Writer.WriteRaw("rn");
                                }
                            }
                            Writer.WriteEndElement();
                            LevelCount++;
                            LineCount = 0;
                        }
                    }
                }
                Writer.WriteEndElement();
            }
            Writer.WriteEndElement();
            Writer.WriteEndDocument();
            Writer.Close();

It does:

  • Convert Unicode punctuations to their ordinary counterpart. For example “ and ” will be converted to “
  • Trim and combine different lines from different paragraphs to they’ll fit into the line format of KTouch. KTouch does not work well with long lines (you can’t see much toward the end of the line) so by default the line is trimmed at 80 characters

For a text source to practice, I decided to drop by project Gutenberg (FYI they digitize out-of-copyright books). They offer a range of about 3000 books in a dozen of languages to choose from. The books are mostly in plain text format, which is just perfect for this purpose.

The first novel I have chosen to type is Godfather by Mario Puzo 🙂 If you want to have a ride too then here’s the lesson maker’s source and here’s the lesson file for Godfather (note that due to bugs in KTouch, some characters toward the end of each lesson will disappear). The lesson file has over 2000 levels available for practicing.

Life with Ubuntu

First, Happy new year to you and your family 🙂

I have been using Linux on a daily basis for several months now and so far life is quite comfortable. For the first time I can simply forget what people have been telling me  since day one –  “you always need antivirus protection”. Heh, how the hell viruses is gonna infect my computer when even I myself can’t modify the system files! Yes, it’s true that if enough people use it there will be some folks spend time to carve into each input box in the whole operating system to find exploits but considering the rate of Firefox adoption over IE, I’d say it may took decades before Linux can take over. This world is resistant to changes, even if it’s good.

Though the default Ubuntu distribution works out of the box and satisfy the needs of most user, it’s oriented to the naive, who doesn’t have the need for more complex configurations, say – multilingual inputs. That’s when Google comes in handy but it may take a while and some searching skills to find what you need. There’s many question and answer in the Ubuntu forums but they are not always anwered satisfactorily.

Below is a collection of what I have to Google for the past months.

Japanese input

I posted about Japanese on Windows back from last year but I don’t really need Japanese for Linux until now – when my HDD is somewhere in the middle of snowstorms in America and I’m working on a little USB flash drive. It kinda feels like a netbook except it’s faster 😛

Windows doesn’t differentiate between input language and keyboard layout –  it combines both into the IME environment. The result is hideous registry settings to configure what should be readily available. It’s a different thing on Kubuntu: the keyboard layout you can configure in System Settings > Regional & Language > Keyboard Layout is for your physical keyboard only, which means if tells the operating system if there’s something special about your keyboard, like is it laid out in Dvorak, does it have extra function keys, does it have the Japanese switch button etc. To input languages you have to use the input method (IME) which is what translates what you type on the keyboard to something else according to configuration. For example, the Vietnamese input method translates Tie61ng Viet65 into “Tiếng Việt” and the Japanese input method translate katakana into “カタカナ”. All this happens within an input framework. To put it simple, an input framework is a tool that let you switch between languages (This is not technically correct but you’ll know better when you have become familiar with it). Being an operating system of choices, there’s three framework for you to choose on Linux: UIM, SCIM and iBus with iBus being the future.

To install iBus, first add the following line to /etc/apt/sources.list

deb http://ppa.launchpad.net/ibus-dev/ibus-1.2-karmic/ubuntu karmic main #IBus 1.2 for Karmic

Then run

sudo apt-get install ibus ibus-gtk

Once iBus has been installed, you can install input languages, say Vietnamese and Japanese:

sudo apt-get install ibus-unikey ibus-anthy

You can activate iBus with

im-switch -s ibus

It requires a restart and then you are good to go! 🙂 Note that instead of using the command line apt-get, you can type “ibus” into the software installer to find the packages and install them.

You can see the language you want to type is not bound to any specific keyboard layout here – at the start of this post you can have Qwerty, Dvorak or Colemak – that doesn’t matter, you can use it to input the language you have installed!

References: ibus @ ubuntu vn, ibus project

Multilingual input with Opera 10

Opera is a great browser, many of the functionality that defines the modern browser is from opera – say tabbed browsing, integrated search function, modular design and many others. Should Opera be free from the start it could have overthrown IE as the most popular browser. Well, what’s history is history already 🙂

I use Opera on linux for the sake of simplicity – it provides all the functions I need without the need to install any fancy plugin (I’m talking about you, Firefox). It’s a surprise that I can’t use iBus to type Vietnamese in Opera when I first installed it :(.

I found out why: the default version you can download from Opera’s homepage is compiled with the Qt3 library while iBus works with Qt4 only. Luckily Opera is also compiled with Qt4 but you’ll need to cruise around some FTP servers to get it.  Click here to get version 10.10 for ubuntu/x86, which is the latest at the time of writing.

When 128MB isn’t enough or how to expand your Ubuntu installation on a USB stick

Kubuntu 9.10 comes with a new feature: make a USB stick bootable and persists your changes between sessions. At first you may think the default storage space of 128 MB could be enough but after some themes, customizations and application s (namely Firefox); 128MB is used up in no time.

Your Linux home is stored within an image file in the root of the drive – casper-rw. You can allow Linux to use a bigger share of your stick by expanding the file then imposes ext3 formatting on it with the following steps:

# dd if=/dev/zero bs=1M count=1024 >> casper-rw
# e2fsck -f casper-rw
# resize2fs -f casper-rw
# e2fsck -f casper-rw

1024 means 1024MB, change this to whatever size you want. Note that this capacity is added to your existing quota, so if you have 128MB already then after this command you have 1153MB for storage.

File managing – Norton Commander style

While Kubuntu have dolphin as a pretty good file manager, iallowsng you to split windows and drag the navigation dock around, it’s still get nowhere close the original NC feel (say tabbing, F3, F4 and F5). Of course Midnight Commander have been around since Linux was still stuck in the server rooms but what? A text mode program on the all cute KDE Oxygen? Does not sound right to me.

Good thing that there’s Krusader. It’s a bit old but seems to work fine. The menu bar, button bar and command line stays true to the tradition – I feel like home 🙂