Debug information for distribution builds

The Protocol: Release binaries should not have debug information with them to prevent malicious users from reverse engineering the code.

Issues: When issue comes in field from customer it becomes difficult for the developer to crack the issue without proper debug symbols.

Resolution:
1. Generate distribution builds with debug information.
2. Separate the debug information from release builds and store them for future in a file.
3. Ship the stripped binary.
4. For external issues use the debug information separated earlier to debug the issue.

Windows (PDB)
With Visual Studio (Windows) its easy. VS provides the facility to create Program Database (PDB) files to store the debug information.

A program database (PDB) file holds debugging and project state information that allows incremental linking of a Debug configuration of your program. A PDB file is created when you compile a C/C++ program with /ZI or /Zi.

In Visual C++, the /Fd option names the PDB file created by the compiler.

Example:
cl.exe /Zi /Fd"Test.pdb" Test.cpp

Linux (objcopy)
In Linux objcopy command can be used to achieve the same results.
1. Generate distribution builds with debug information
Use compiler flag -g while building the libs and binaries.

2. Separate the debug information from release builds and store them for future in a file.
Run objcopy --only-keep-debug test.dbg to create a file containing the debugging info.

3. Remove the symbol information from the executable
Run objcopy --strip-all test to create a stripped executable.

4. Creates a .gnu_debuglink section which contains a reference to debug file in executable
Run objcopy --add-gnu-debuglink=test.dbg test to add a link to the debugging info into the stripped executable.

Generating hang dumps in Linux (gcore)

Last week encountered a process hang on Linux; well the need arises to take the dump of the process. You can generate the dumps of a running process on Linux in two ways.

1. gcore - The gcore utility creates a core image of the specified process.

gcore [-s] pid
-s Stop the process while gathering the core image, and resume it when done. This guarantees that the resulting core dump will be in a consistent state. The process is resumed even if it was already stopped.

$ gcore -s 547
...
...
Saved corefile core.547

2. Using gdb

First attach gdb to the running process.
$ gdb - 547 # note the - has spaces on both sides.
Or
You can use gdb's attach command.
$ gdb
(gdb) attach 547

Once you are attached to the running process, you can take the dump using generate-core-file(gcore) command.
(gdb) generate-core-file
Saved corefile core.547

In memory FileSystem

Linux(tmpfs and ramfs)
In Linux, you can convert part of your physical memory to be used as a disk partition. In this partition you can read/write files just like any other file on disk, but as the read/writes are done in memory they are really fast.

This partition is useful for applications which at run-time create lots of temporary files & have high IO on these files. For example, installers, extractor tools.

In Linux, we have tmpfs and ramfs mounts that provide the power to create in-memory file system for fast reading and writing files from and to the primary memory.

How to create and mount tmpfs

# mkdir -p /mnt/tmpfs

# mount -t tmpfs -o size=200m tmpfs /mnt/tmpfs
This will create a tmpfs partition of 200MB.

You can view the partition with
# df -k
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda3 125641908 99794556 19515336 84% /
tmpfs 1557024 40000 1517024 3% /lib/modules/2.6.24-27-generic/volatile
tmpfs 204800 0 204800 0% /tmp/tmpfs

How to create and mount ramfs

# mkdir -p /mnt/ramfs

# mount -t ramfs -o size=200m ramfs /mnt/ramfs

You can view the partition with
# mount

/dev/sda3 on / type ext3 (rw,relatime,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
/sys on /sys type sysfs (rw,noexec,nosuid,nodev)
varrun on /var/run type tmpfs (rw,noexec,nosuid,nodev,mode=0755)
varlock on /var/lock type tmpfs (rw,noexec,nosuid,nodev,mode=1777)
udev on /dev type tmpfs (rw,mode=0755)
devshm on /dev/shm type tmpfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
lrm on /lib/modules/2.6.24-23-generic/volatile type tmpfs (rw)
securityfs on /sys/kernel/security type securityfs (rw)
none on /proc/fs/vmblock/mountPoint type vmblock (rw)
tmpfs on /lib/modules/2.6.24-27-generic/volatile type tmpfs (rw,mode=0755)
tmpfs on /tmp/tmpfs type tmpfs (rw,size=200m)
ramfs on /mnt/ramfs type ramfs (rw,size=200m)


ramfs vs tmpfs
Basically the same with minor differences:
* ramfs can grow dynamically, tmpfs can not.
This means that in ramfs you can keep on writing beyond the maximum size of your partition, the system will not stop you. So you need your application to keep track of the size.
Where as, tmpfs will not grow dynamically. It may give errors similar to “No space left on device”, when you hit the maximum size of partition.

* tmpfs uses swap, ramfs doesn't.

* Both are volatile file system. So on system shutdown/crash you lose the data.

Microsoft Windows
Windows systems have a rough analog to tmpfs in the form of "temporary files". Files created with both FILE_ATTRIBUTE_TEMPORARY and FILE_FLAG_DELETE_ON_CLOSE are held in memory and only written to disk if the system experiences low memory pressure. In this way they behave like tmpfs, except the files are written to the specified path during low memory situations rather than swap space.

Which folders are shared from my computer?

I had one shared folder that had kept me wondering for sometime. I was not able to find it on my disk.
Actually you can see all you shared folders, with their location using 'Computer Management' utility.

1. Right click My Computers
2. Select Manage option.
3. Go to Shared Folders.
4. Then go to Shares.

OR
use Start->Run-> fsmgmt.msc

There you are.

Using this I found that my drives are also shared. This is a problem if someone knows that they can access the computers drives, they can get hold of all your information and data.
I disabled the sharing of drives, using the same utility.

Dear Norton,
1. Don't block my network and ask me to provide a remote access.
2. For a change, install successfully in one shot.
3. Reimburse the money I pay because of you, for my system's repair.
4. Never, never, never slow down my system boot.
5. Update the software, but don't stop me from working.
6. Make the subscription possible.
7. Provide efficient support, if any. Else declare you have none.
8. You know there are free AV programs available, that are better than you.

Building Pthread static library for VS2005 pthread

In case you're trying to find a way to use pthreads-win32 as a static lib

Build pthread

1. Get the pthread source code from: http://www.sourceware.org/pthreads-win32/
2. Define PTW32_STATIC_LIB to build pthreads. 

Be careful to use the same setting for code generation in the lib generation and your project - Debug Multithread DLL (/MDd) or Multithread DLL (/MD).
This will avoid any error with “_errno not found”

Build your application

1. Define PTW32_STATIC_LIB in your project which will use pthreads (because this define will change the behaviour of pthreads.h file). This will avoid any error msg with “dll implicit…” 
2.  When pthreads is used as a DLL, the code in dll.c is execute when the dll is loaded. So when using as a static lib, you'll need to call these code also during the initialization and destruction of your process: pthread_win32_process_attach_np(); and for release: pthread_win32_process_detach_np(); This will avoid the crash when you call pthread_create.

Aaahhh!! Warnings and Errors in Visual Studio.

1. Project : error PRJ0003 : Error spawning 'mt.exe'.
Solution:
Change MSVS 2005 options (Tools menu > Options > Project and Solutions > VC++ Directories) to ensure that
$(SystemRoot)
$(SystemRoot)\System32
$(SystemRoot)\System32\wbem
are specified BEFORE $(PATH).

2. Warning LNK4006: __NULL_IMPORT_DESCRIPTOR already defined in XYZ.lib
Solution:
Don't put any 'Additional dependencies' .libs in your static libraries. Only add them as 'Additional dependencies' to your .exe. If you are using someone elses library that does that, tell them to stop it.

3. warning C4714: function 'XYZ()' marked as __forceinline not inlined
Solution:
http://msdn.microsoft.com/en-us/library/a98sb923.aspx

Topcoder is a well known competition arena for the algorithm geeks. Now we have got some tools that help us parse the problem definition on the topcoder and generate template code for the problems, and the test cases too.

Well I checked and played around many such tools. To list a few:

1. Eclipse coder (recommended).
2. KawigiEdit
   
3. Topcoder Auto Gen

Eclipsecoder is a cool plugin for the eclipse platform. It can generate code for you in C++/Java not in C. Now as the tool is very new it crashes sometimes, but comparing the advantages it is nothing.
The main problems that I faced were with Http tunelling option in it. It was not allowing me to check mark it. The problem seem to be of the arena applet. Seeing the posts on the forums a guy got the same problem and got rid of it by making a fresh installation of eclipse and Eclipsecoder plugin. Well it didn't worked for me. Still fighting with it. [will post the solution soon.]

In Eclipsecoder you can also play on the problem archive. It is really cool feature. In it you can work around with the submissions too.

Now in KawigiEdit the editor is for Topcoder. Got a good parser, with topcoder's GUI. The coder generated is also nice. The recommended one, if you don't want to use Eclipse feature..

Topcoder AutoGen is a windows-only tool. Easy to use but does not have a good parser. Usually misses the tricky test cases.