Home Download News FAQ / Knowledge Base Screenshots Documentation Support
philosophical imaginary
Table of Contents


Whence you're in total trouble, a debugger is the way to help out. It is also the proper tool to report information back to the citadel development team (like Mozilla will do with its talkback facility). There are several instances of debuggers around, mdb / dbx for Solaris for example, but the one we will talk about here is the in the open source world most used GDB that you can also operate through DDD, KDevelop, Emacs or plain through its command line, which all of the above front ends also offer. We will show some basic operations here, you may want to search the gdb man page if you want to go deeper into detail.
Please remember that its always a good idea to check whether your troubles persist with the SVN Head revision (after doing a backup of your data).

Which kind of binaries produce useful output?

By default your binaries installed to your system are stripped to safe system resources. You can find it out like that:

# file /usr/sbin/citserver 
/usr/sbin/citserver: ELF 32-bit LSB executable, Intel
80386, version 1 (SYSV), for GNU/Linux 2.4.1, dynamically
linked (uses shared libs), stripped

These will produce allmost meaningless output in gdb, just maybe the addr2line utility can gather some information from that. However, the better method is to use an unstripped binary (the install process does this), that you will find inside your source tree. If you like to do stepping and not want gdb to jump back and forth, compile it with -ggdb -o0 compiler flags.

Citadel brings you untripped/citserver, which matches your active citserver. Use this to analyse your core files.

How do I make my system produce core-files?

On Linux this can be achieved with the ulimit command which is documented in the bash man page.

ulimit -c 999999

The core files are created in the working directory. Be careful, they can grow and eat up your disk space. Making the whole system create core files on crashes in a central folder can be achieved like that:

ulimit -c unlimited
echo 1 > /proc/sys/kernel/core_uses_pid
echo /tmp/core-%e-%p-%t > /proc/sys/kernel/core_pattern

Analyzing Core files

If a program crashes due to illegal commands its state may be conserved into a core-dump. You can use this core dump with gdb to find out at which point it ceased operation.

gdb citserver core dump

From now on you can do all data examination you could do if you just stopped the program.

thread apply all bt

will show you a stack trace of the functions in the citadel binary that were active. If you just want to report the bug, please use the output of

thread apply all bt full

If you want to save the contents of a pointer, you can do it like that:

dump binary memory /tmp/decoded  decoded &decoded[10000]

would save the found 10k bytes after the place decoded points to.

Running Programs in GDB

Setting a breakpoint can be done with

break main

for example. start it with

run -x7

(which will pass -x7 to the debugged process as commandline parameter) and it will stop at the start. you can make it step forward with


and enter a function with


print variables with

print varname

or on every step with

display varname
handle SIGPIPE nostop

will stop gdb from stopping on other programs disconnecting from a citadel server.

Using Valgrind

Nobody's perfect. Valgrind is the most decent tool around these days to detect tiny flaws in programs and bust memory leaks. (the process growing constantly because of it forgets about memory it demanded from the system) We use valgrind to ensure the quality of citadel. This is how you can run it:

valgrind --log-file=/tmp/valgrindlog \
   --show-reachable=yes --leak-check=full citserver 



Sometimes code doesn't work as expected in special situations, a program develops hotspots. Profilers can help here, gprof does in our case. It needs special binaries, heres how to create them: If you want to create them via the deb-build, just do

export DEB_BUILD_OPTIONS="debug profiling"

If you want to configure and compile yourself, do

CFLAGS = -O0 -ggdb -rdynamic -MD -MP

and add --with-gprof to your configure files


Now you're equipped with those special binaries, run them, produce the high load situation you've experienced, and stop them regularly afterwards, so they can write their 'gmon.out' file containing the profiling raw data. WebCit can be shut down with 'killall -HUP webserver', citserver will finish its job with 'sendcommand down'.

Both of them will write a gmon.out into their working directory:

DebsEasy Install
Copyright © 1987-2020 Uncensored Communications Group. All rights reserved.     Login (site admin)