Problem with Qt compilation under Windows 64-bit

This short post will show you how to resolve following error during Qt compilation on 64-bit windows:

Creating library ..\..\..\..\lib\QtWebKitd4.lib and object ..\..\..\..\lib\Qt
WebKitd4.exp
PluginViewWin.obj : error LNK2019: unresolved external symbol _HBeginPaint refer
enced in function "private: static struct HDC__ * __cdecl WebCore::PluginView::h
ookedBeginPaint(struct HWND__ *,struct tagPAINTSTRUCT *)" (?hookedBeginPaint@Plu
ginView@WebCore@@CAPEAUHDC__@@PEAUHWND__@@PEAUtagP AINTSTRUCT@@@Z)
PluginViewWin.obj : error LNK2019: unresolved external symbol _HEndPaint referen
ced in function "private: static int __cdecl WebCore::PluginView::hookedEndPaint
(struct HWND__ *,struct tagPAINTSTRUCT const *)" (?hookedEndPaint@PluginView@Web
Core@@CAHPEAUHWND__@@PEBUtagPAINTSTRUCT@@@Z)
..\..\..\..\lib\QtWebKitd4.dll : fatal error LNK1120: 2 unresolved externals
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 9.0\
VC\BIN\x86_amd64\link.EXE"' : return code '0x460'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 9.0\
VC\BIN\nmake.exe"' : return code '0x2'

This error is caused by bug in QMAKE_HOST variable used in WebCore.pro file. This variable is tested to determine destination platform (x86 or x64).
This variable contains x86 value also on 64-bit builds instead of x86_64 value. This value is tested on the following line in a WebCore.pro file:

win32:!win32-g++*:contains(QMAKE_HOST.arch, x86_64):{

If you want to fix this error, simply replace QMAKE_HOST by QMAKE_TARGET variable. Updated line should look like this:

win32:!win32-g++*:contains(QMAKE_TARGET.arch, x86_64):{

After this update all required asm defines are correctly included and build of WebKit will be successful.

Update for this error

In newer versions of Qt there is already applied fix the error with QMAKE_HOST / QMAKE_TARGET, but also there is a new problem ;-(.

The Problem is in the following “if” definition:

   if(win32-msvc2005|win32-msvc2008):equals(TEMPLATE_PREFIX, "vc") {
        SOURCES += \
            plugins/win/PaintHooks.asm
    }

When you compile your project for MSVC 2010, .asm file isn’t included. It’s necessary to add win32-msvc2010 to this definition. So updated WebCore.pro for Qt 4.8 will look like this:

 if(win32-msvc2005|win32-msvc2008|win32-msvc2010):equals(TEMPLATE_PREFIX, "vc") {
        SOURCES += \
            plugins/win/PaintHooks.asm
    }

Next problem solution

Next problem which I found during compilation is wrongly set QMAKE_TARGET.arg. For unspecified reason, when I compile 64bit library in 64bit system in 64bit VS command line, this variable has value x86 instead of x86_64.
The problem is in the qmake project.cpp file. QMAKE trying to determine 64/32 bit version based on the %PATH% configuration. Qmake search for VS-PATH\bin\amd64 of \bin\x86_amd64 string. Unfortunately there is a bug in concating searched string. This is how looks original project.cpp file at line 3177:

QString vcBin64 = qgetenv("VCINSTALLDIR").append("\\bin\\amd64");
QString vcBinX86_64 = qgetenv("VCINSTALLDIR").append("\\bin\\x86_amd64");

The problem is, that VCINSTALLDIR contains \ at the end of the string, and then we append \ again. So, result is:

QString vcBin64 = qgetenv("VCINSTALLDIR");
if ( vcBin64.at( vcBin64.size()-1 ) != '\\' )
  vcBin64.append("\\");
vcBin64.append("bin\\amd64");

QString vcBinX86_64 = qgetenv("VCINSTALLDIR");
if ( vcBinX86_64.at( vcBinX86_64.size()-1 ) != '\\' )
  vcBinX86_64.append("\\");
vcBinX86_64.append("bin\\x86_amd64");

Now, QMake correctly determine 32/64 bit compilation and then correctly include PaintHooks.asm and your Qt WebCore will be sucesfully compiled.

How to parse Apache Log and insert data to MySQL database

#!/bin/bash

echo Generating new data from log
echo "#generated sql file" > data.sql

echo "truncate table log_data;" >> data.sql
echo "START TRANSACTION;" >> data.sql

FIND_PATTERN="\
\(\S*\)\s*\
\(\S*\)\s*\
\(\S*\s*\S*\)\s*\
[\x5B]\([^\x5D]*\)[\x5D]\s*\
\"\([^\"]*\)\"\s*\
\(\S*\)\s*\
\(\S*\)\s*\
\"\([^\"]*\)\"\s*\
\"\([^\"]*\)\"\s*\
";

REPLACE_PATTERN="\
INSERT INTO log_data \
(domain,ip,other,date,url,status,size,referral,clientstring) \
VALUES \
('\1','\2','\3','\4','\5','\6','\7','\8','\9');";

APACHE_FILE="other_vhosts_access.log";

if [ -f $APACHE_FILE.size ];
then
   ALREADY_PROCEED=`cat $APACHE_FILE.size`
   echo "Already proceed size ($APACHE_FILE): "$ALREADY_PROCEED;
else
   ALREADY_PROCEED=0;
   echo "New log file $APACHE_FILE";
fi

FILTER_PATTERN="\.css|\.ico|\.png|\.jpg|\.txt|\.js|\.gif|\.swf"

#echo $FIND_PATTERN"/"$REPLACE_PATTERN;

cat $APACHE_FILE |
tail -c +$ALREADY_PROCEED | #preskocime jiz zpracovane radky
#head -5000 | #debug
#head   -1128863 | #tail -2  |
#tail -n +1177103 | head -3 |
#sed -r "s/(\(.*\))//g" | #debug - odebere z clientstring (...)
#sed  "s/^[^+]*//g" | #debug - odebere pocatek stringu az do GET
#sed -r "s/Mozilla|Gecko|Firefox//g" | #debug odebere tyhle klicovy slova
sed -r "s/^(subdomain|www)\.//"   | #odebereme prefix subdomain. a www.
sed -r "s/^([^:]*)(:[0-9]*)/\1/"  | #odebereme port
sed -r -n "/^(orm-|inventic).*/p" | #exportujem jen nektere domeny
sed "s/\([\x5c][\x22]\)//g"       | #odebrani \" retezcu
sed "s/\([']\)//g"                | #odebrani apostrof ktery delaj pak bordel v$
sed -r "s/ \"GET ([\x2F].*) HTTP[\x2F][0-9.]*/ \"\1/" | #dame pryc GET a HTTP1.1
#sed -r -n  "/$FILTER_PATTERN/p" #| debug- vypise filtrovane .css/.png/...
sed -r "/$FILTER_PATTERN/d"       | #vyfiltrujeme pryc .css, .png atd
#sed  "/^$FIND_PATTERN$/d"        | #debug - toto lze pouzit na vypis radku kte$
sed -n "/^$FIND_PATTERN$/p"       | #vypiseme vsechny radky ktere jsou validni
sed "s/$FIND_PATTERN/$REPLACE_PATTERN/" >> data.sql

echo "COMMIT;" >> data.sql

#ulozime velikost zpracovaneho logu
stat -c"%s" $APACHE_FILE > $APACHE_FILE.size
#vlozime data do Mysql
echo Putting data to MySQL
mysql \
  --user=root \
  --password=1234 \
  --host=192.168.0.10 \
  --database=server_log < data.sql

SandyBridge

CPU

http://www.alza.cz/intel-core-i7-2600k-quad-core-sandy-bridge-d201345.htm

http://www.alfacomp.cz/php/product.php?eid=1051400830984YJ12YG

http://interlink.tsbohemia.cz/intel-core-i7-2600k-3-40ghz-8mb-box-lga1155-sandy-bridge-s-integrovanou-grafikou-odemceny-nasobic_d116932.html

http://www.czechcomputer.cz/product.jsp?artno=85286

http://www.mironet.cz/intel-core-i72600k-quad-core–34-ghz–8mb-cache–1155–box–overclocking+dp117723/

MB

http://interlink.tsbohemia.cz/komponenty-it-zakladni-desky-desktopove-intel-bez-vga-socket-1155_c10856.html

http://www.alfacomp.cz/php/product.php?eid=10514008M0984YC12H4

http://www.czechcomputer.cz/product.jsp?artno=84646

http://www.alza.cz/asus-p8p67-pro-d202981.htm

RAM

http://interlink.tsbohemia.cz/dimm-ddr3-4096mb-2000-dc-kit-2×2048-transcend-9-9-9-24-axeram_d94573.html

http://interlink.tsbohemia.cz/dimm-ddr3-4096mb-2400-dc-kit-2×2048-transcend-9-11-9-28-axeram_d109507.html

http://www.alza.cz/a-data-4gb-kit-ddr3-1600mhz-cl7-xpg-series-d214745.htm

MONITOR

http://www.alza.cz/lcd-monitory/24-a-vetsi-lcd/18843375.htm#CatId=18843375&limit=&prod=1396,1287,&pn=&Pg=0


Collection of links about ORM, Datamapper and design patterns

Domain model

http://www.slideshare.net/weierophinney/architecting-your-models

ORM design patterns

http://martinfowler.com/eaaCatalog/mapper.html
http://martinfowler.com/eaaCatalog/dataMapper.html

DataMapper

http://stackoverflow.com/questions/207758/data-mapper-pattern
http://stackoverflow.com/questions/762252/fowler-data-mapper-object-creation
http://stackoverflow.com/questions/4658340/should-data-mapper-reference-domain-model
http://stackoverflow.com/questions/1977684/should-a-finder-method-be-part-of-the-data-mapper-or-part-of-the-domain-class
http://stackoverflow.com/questions/3738687/using-the-data-mapper-pattern-should-the-entities-domain-objects-know-about-th

Anemic domain model

http://www.martinfowler.com/bliki/AnemicDomainModel.html (Anemic domain model – why huge service layer  is bad)
http://stackoverflow.com/questions/258534/anemic-domain-model-pros-cons

Data Mapper best practices

http://stackoverflow.com/questions/4465237/doctrine2-best-practice-should-entities-use-services
http://stackoverflow.com/questions/4108291/using-entitymanager-inside-doctrine-2-0-entities/4109458#4109458

Doctrine2 articles

http://www.doctrine-project.org/docs/orm/2.0/en/cookbook/aggregate-fields.html

Qt programming tips

How to configure mkspec for paralel builds on Windows

Edit qmake.conf file in Qt\mkspecs\win32-msvcXXXX. Add -MP2 / -MP4 to the following defines:

QMAKE_CFLAGS_RELEASE    = -O2 -MD -MP4
QMAKE_CFLAGS_RELEASE_WITH_DEBUGINFO += -O2 -MD -Zi -MP4
QMAKE_CFLAGS_DEBUG      = -Zi -MDd -MP4

Mount error(12): Cannot allocate memory

Problem

When mounting some windows samba drives, “mount error(12)” error can occurred. Problem is in windows samba server settings. Full linux error message:

mount error(12): Cannot allocate memory
Refer to the mount.cifs(8) manual page (e.g.man mount.cifs)

and error message in windows event log:

Source: srv
Event ID: 2017
Level: Error
The server was unable to allocate from the system nonpaged pool because the server reached the configured limit for nonpaged pool allocations.

Solution

Update following registry:

HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\LargeSystemCache = 1
HKLM\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters\Size = 3

After you update these values, restart your computer.

More about this issue:

http://alan.lamielle.net

How to deploy Qt application on MacOs

Missing dylib libraries

If your application using some externals dynamic libraries (called .dylib in MacOs system), you have to provide all these libraries. The simplest way is to provide these libraries into the Application.app directory.

Find out which libraries are used

Qt provide really useful tool called “otool” which serves to lots of purpose. One of them is to detect which shared libraries are required by our application. For this purpose otool has parameter -L. The usage is following:

otool -L ./Test1.app/Contents/MacOS/Test1

Output could looks like this:

./Test1:
        libboost_iostreams.dylib (compatibility version 0.0.0, current version 0.0.0)
        libboost_filesystem.dylib (compatibility version 0.0.0, current version 0.0.0)
        libboost_system.dylib (compatibility version 0.0.0, current version 0.0.0)
        libboost_thread.dylib (compatibility version 0.0.0, current version 0.0.0)
        libboost_date_time.dylib (compatibility version 0.0.0, current version 0.0.0)
        libboost_regex.dylib (compatibility version 0.0.0, current version 0.0.0)
        /Users/dev/dev/SharedLibraries/libiconv/lib/libiconv.2.dylib (compatibility version 8.0.0, current version 8.0.0)
        /Users/dev/dev/ExternalLibraries/../SharedLibraries/libxml/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.8.0)
        /Users/dev/dev/ExternalLibraries/../SharedLibraries/libxslt/lib/libxslt.1.dylib (compatibility version 3.0.0, current version 3.26.0)
        /Users/dev/dev/ExternalLibraries/../SharedLibraries/libxslt/lib/libexslt.0.dylib (compatibility version 9.0.0, current version 9.15.0)
        /Users/dev/dev/ExternalLibraries/yaml-cpp-0.2.5/build/libyaml-cpp.0.2.dylib (compatibility version 0.2.0, current version 0.2.5)
        QtGui.framework/Versions/4/QtGui (compatibility version 4.7.0, current version 4.7.0)
        QtCore.framework/Versions/4/QtCore (compatibility version 4.7.0, current version 4.7.0)
        /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.9.0)
        /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 625.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.0)

Output from “otool” is simmilar to linux tool “ldd”.

The Mac Deployment Tool

Because manual copying and configuring all shared libraries to our application could be difficult, Qt have deployment tool which do all work for us. Tool is called “macdeployqt” and can be found in /Developer/Tools/Qt/macdeployqt.

 /Developer/Tools/Qt/macdeployqt ./GeneratedFilesMacOs/Debug/Test1.app

Note: Every time when you call macdeployqt tool, project have to be build from scratch. But is sufficient when whole result directory is removed and linked again:

rm -rf ./GeneratedFilesMacOs/Debug/Test1.app/
make

g++ template method problem

Quick post about problem which I already solved, but think that could be handy for someone else.

I have following code:

template<class T>
struct Test
{
	T val;
	template <class T2>
	void DoSomething( T2 &obj )
	{
		obj.FindByType<int>();
	}
};

When compiling under Visual studio, everything is ok. But when try the same code snippet under g++, have following error:

test.cpp: In member function ‘void Test<T>::DoSomething(T2&)’:
test.cpp:48: error: expected primary-expression before ‘int’
test.cpp:48: error: expected ‘;’ before ‘int’

This is because compiler doesn’t known FindByType method, because T2 is templated argument. The solution which I found is in adding keyword template before FindByType method name. So updated source code will look like this:

template<class T>
struct Test
{
	T val;
	template <class T2>
	void DoSomething( T2 &obj )
	{
		obj.template FindByType<int>();
	}
};

After this update, code will be compiled correctly under both compilers.

Git tips

Show files status

git status #show all files
git statis -u -s #show untracked in short listing

Commit with message

git commit -m"text"

Recursive add files by mask to git

git add *.mask -A

Linux-recursive remove some files

find -iname '*mask' | xargs rm

How to create global .gitignore file

git config --global core.excludesfile /file-path/.gitignore-global

TENG – c++ templating engine

Project site: http://teng.sourceforge.net/?page=home

External project documentation: http://teng.olmik.net/

Latest TENG source code: http://teng.cvs.sourceforge.net/teng/ (Note: Source code referenced from main site isn’t latest! )

How to compile TENG on windows

Compilation under Windows is possible only using MinGW and with few modifications in TENG code (because there is few glitch which didn’t meet c++ standards). If you want more information or updated TENG version, please let me know.