Notarized Qt binaries for OSX

Notarization sh script for automate whole process. Script is created based on several examples found on the web. Links to these examples can be found bellow (mostly thanks to Logcg.com blog).

Notarize DMG file

#!/bin/sh

APPLICATION_PATH=$1
UNDLE_ID="___BUNDLE_ID____"
APPLE_USER="___APPLE_USER____"
APPLE_PASSWORD="___APLE_APPSPECIFIC_PASSWORD___"

 
echo "Running notarize-app command for file $APPLICATION_PATH"
xcrun altool --notarize-app -t osx -f "$APPLICATION_PATH" --primary-bundle-id=$BUNDLE_ID -u $APPLE_USER -p $APPLE_PASSWORD &> notarize-status.txt

echo "Notarize-app complete. Result: "
cat notarize-status.txt

uuid=`cat notarize-status.txt | grep -Eo '\w{8}-(\w{4}-){3}\w{12}$'`
echo "Request UUID is $uuid"

while true; do
    echo "checking for notarization..."
    xcrun altool --notarization-info "$uuid" --username $APPLE_USER --password $APPLE_PASSWORD &> notarize-response.txt
    
    echo "Response:"
    cat notarize-response.txt
    
    t=`cat notarize-response.txt | grep "success"`
    f=`cat notarize-response.txt | grep "invalid"`
    if [[ "$t" != "" ]]; then
        echo "notarization done! Stampling application $APPLICATION_PATH"
        xcrun stapler staple "$APPLICATION_PATH"
        echo "stapler done!"
        break
    fi
    if [[ "$f" != "" ]]; then
        echo "$r"
        return 1
    fi
    echo "not finish yet, sleep 30sec then check again..."
    sleep 30
done

Notarize .app / .zip

#!/bin/sh

APPLICATION_PATH=$1
ZIP_PATH=./application-to-notarize.zip
BUNDLE_ID="___BUNDLE_ID____"
APPLE_USER="___APPLE_USER____"
APPLE_PASSWORD="___APLE_APPSPECIFIC_PASSWORD___"


echo "Packing app $APPLICATION_PATH to zip file $ZIP_PATH"
ditto -ck --rsrc --sequesterRsrc $APPLICATION_PATH $ZIP_PATH
 
echo "Running notarize-app command for file $ZIP_PATH"
xcrun altool --notarize-app -t osx -f "$ZIP_PATH" --primary-bundle-id=$BUNDLE_ID -u $APPLE_USER -p $APPLE_PASSWORD &> notarize-status.txt

echo "Notarize-app complete. Result: "
cat notarize-status.txt

uuid=`cat notarize-status.txt | grep -Eo '\w{8}-(\w{4}-){3}\w{12}$'`
echo "Request UUID is $uuid"

while true; do
    echo "checking for notarization..."
    xcrun altool --notarization-info "$uuid" --username $APPLE_USER --password $APPLE_PASSWORD &> notarize-response.txt
    
    echo "Response:"
    cat notarize-response.txt
    
    t=`cat notarize-response.txt | grep "success"`
    f=`cat notarize-response.txt | grep "invalid"`
    if [[ "$t" != "" ]]; then
        echo "notarization done! Stampling application $APPLICATION_PATH"
        xcrun stapler staple "$APPLICATION_PATH"
        echo "stapler done!"
        break
    fi
    if [[ "$f" != "" ]]; then
        echo "$r"
        return 1
    fi
    echo "not finish yet, sleep 30sec then check again..."
    sleep 30
done

Useful links:

Incredibuild and “cannot create temporary il file”

This is caused by latest MS security update. After applying these two 3126587 and 3126593 previous versions of Incredibuild stopped working.

Command line error D8037 : cannot create temporary il file 

The solution is to uninstall it

https://support.microsoft.com/en-us/kb/3126587
https://support.microsoft.com/en-us/kb/3126593

On Windows 10 the responsible update is KB3135173

Next steps

Also it’s necessary to disable these updates in auto-update!

explorer_2016-04-04_10-02-20

More info:

http://xoreax.helpserve.com/Knowledgebase/Article/View/131/8/command-line-error-d8037–cannot-create-temporary-il-file

QML Notes

Sources

F5 for instant reload,

 

class UltraView : public QQuickView {
 protected:
    void keyPressEvent(QKeyEvent*event) override {
         if(event->key()==Qt::Key_F5){
             auto oldSource=source();
             setSource({});
             engine()->clearComponentCache();
             setSource(oldSource);
         }
    }
};

F10 and F11 for slowing down animations

else if(event->key()==Qt::Key_F10) { 
  QUnifiedTimer::instance()->setSlowModeEnabled(true);
  QUnifiedTimer::instance()->setSlowdownFactor(10);
} else if(event->key()==Qt::Key_F11) {
  QUnifiedTimer::instance()->setSlowModeEnabled(false);
  QUnifiedTimer::instance()->setSlowdownFactor(1);
}

QML Live editing

QML Lint

Qml & QtQuick

Qt documentation

QML Canvas

QML States

QML MouseEvents

QML c++ integration

QtQuick painted items

QML charts

QML Diagrams

QML & QtQuick blogposts

QML Selection, multiselection, rectangle-selection

QML Tools

QML Applications

 

Other topics

Qt 5.5.0 an Qt 5.5.1, QWebEngine, QtWebKit and ICU

Notes about compiling QtWebEngine (instead of QtWebkit) together with Qt 5.5.1

Qt 5.5.0 configure switches for ICU:

configure \
    -icu \
    -I Q:\SharedLibraries\icu4c-51_1\icu\include \
    -L Q:\SharedLibraries\icu4c-51_1\icu\lib64

Starting with 5.5.1 it shouldn’t be necessary to compile and ship ICU libraries because ICU should be included in Qt package itself.

Compile QtWebkit

It’s necessary to compile it as standalone module.

cd Qt/qtwebkit
perl Tools\Scripts\build-webkit

Simpler solution

Much easier solution seem to be to use http://www.awesomium.com/  😉

Notes:

  • When changing configure settings, it’s sometimes necessary to perform complete clean rebuild. The  best way how to do that is download completely new source package.
  • nmake clean or nmake distclean unfortunately didn’t work for 100%
  • It’s necessary to have installed Ruby, Python and Perl

Source links:

Other info:

Crash in QTreeWidget / QTreeView index mapping on Mac OSX 10.10 part III

So, one more attempt. Previous articles (part1, part2) mentioned possible solutions to fix crash insinde the QTreeWidget and QAccessibleTableCell.

Unfortunately, deselecting current item still doesn’t fix all issues. The problem with deselection is that it’s not handled correctly via QAccessibleTableCell:

If current index isn’t valid, QAccessible doesn’t correctly update currect QAccessibleTableCell object which caused all this evil crashes.

So there are two ways how to fix it. One is to manually set QAccessibleTableCell but I didn’t find a way how to do that. The second way is to disable QAccessible for Property editor. Unfortunately it’s not an easy task. In QAccessible exists method setActive:

QAccessible::setActive(false)

But this method works only with AccessibleObserver class but not with QPlatformAcessibility which handles real QAcessible::isActive result. Because of this it’s necessary to update setActive method in file qacessible.cpp inside the Qt.

Original method:


void QAccessible::setActive(bool active)
{
  for (int i = 0; i < qAccessibleActivationObservers()->count() ;++i)
    qAccessibleActivationObservers()->at(i)->accessibilityActiveChanged(active);
}

and updated version:

void QAccessible::setActive(bool active)
{
  #ifndef QT_NO_ACCESSIBILITY
    if ( QPlatformAccessibility *pfAccessibility = platformAccessibility() )
      pfAccessibility->setActive(active);
  #endif

  for (int i = 0; i < qAccessibleActivationObservers()->count() ;++i)
    qAccessibleActivationObservers()->at(i)->accessibilityActiveChanged(active);
}

In case you will find better way how to turn off accessibility, please let me know.

But there is still one more problem. Occasionally OS X decides to switch accessible back on:

And this is probably the reason why everything works ok on older OS X but not on 10.10. Because OS X 10.10 randomly turns accessible on which causes this crash. Accessible is usually turned on when opening some modal dialog.

This explains why opening an empty dialog increased the probability to crash the application. The only solution I have found is to disable QAccessible every time the PropertyEditor is cleared up.

QDialog::reject can be invoked multiple times

When user use ESC to close the dialog and press it very quickly two or more times in a row, QDialog::reject can be invoked also multiple times.

This is very dangerous and unpredictable because in case your reject will contain some one-time-execute routine the application will start crashing.

The only solution I found is to add these routines to QDialog destructor instead of to reject/accept.

Qt5 application crashed with error 0xc0000005

This is very interesting crash and I think it should be considered as Qt bug. This problem arise only under very special circumstances. But one after another.

You will identify this problem when your application stopped/exited immediately after the start without any visible error message. It looks like the application isn’t executed at all. When you check application log (Computer -> manage -> Event viewer -> Windows logs -> Application), you will see Error logs:

Windows applications logs

The most interesting part of this log is crash location: ntdll.dll

Faulting application name: Skipper.exe, version: 3.0.1.1120, time stamp: 0x53e9c8d7
Faulting module name: ntdll.dll, version: 6.1.7601.18247, time stamp: 0x521ea8e7
Exception code: 0xc0000005
Fault offset: 0x0002e3be
Faulting process id: 0x1c88
Faulting application start time: 0x01cfb606553e594b
Faulting application path: T:\S2\Skipper.exe
Faulting module path: C:\Windows\SysWOW64\ntdll.dll
Report Id: 98cc8228-21f9-11e4-ab5d-005056c00008

At first sight it seems like some problem inside the windows. But the opposite is true, the problem (as almost always) is inside your app ;-).

As the next step, you can try to debug this executable via Visual Studio to see what happens inside. Simply open executable as project together with .pdb files and execute it. Now you can see that application is correctly executed but crashes as soon as it touches Qt library. The location of crash is inside ntdll.dll in RtlHeapFree() function.

Debuging crash in VisualStudio 2013

So the problem is inside the Qt, right? Almost true, but not for the 100%. When I tried to run this application on computers of my colleagues, everything works ok. So why the application doesn’t work on my computer too?

Resolution

The problem is in new Qt5 plugin system. Besides the common Qt5*.dll files which are loaded immediately after the application start, Qt5 is also loading plugins/platform-plugins dynamically when the application is executed. To locate this plugins, Qt5 uses following method to identify directories where to search for plugins:

QStringList QCoreApplication::libraryPaths()

For some strange reason this library returns as first directory path where Qt5 libraries were compiled and after that location based on the executable. So if your Qt5 path is C:\Qt5, this will be the first path where all plugins are searched for, no matter if the correct version of plugin is located in APP\plugins or APP\platforms. I think this is serious bug in Qt5.

Where is the problem?

And here we’re getting to the core of the whole problem.

If application is compiled on computer with one compiler and used on second  computer which contains the same path to which original computer has installed Qt, the application will load all plugins from your folder instead of itself folder.

In case your computer will contain different version of Qt, different compiler or different platform, application loads incorrect libraries and crashes. Completely, silently and without easy way to determine what’s wrong.

Solution?

The solution is simple, but it isn’t achievable from outside of the Qt library. It would be necessary to Qt as first tried to load libraries from application directory. And only if no plugins were found in application directory, the application would try to search for plugins in Qt directory.

Qt change solution

The simplest way how to fix this issue inside the Qt library would be to rename/update appendApplicationPathToLibraryPaths  function to prependApplicationPathToLibraryPaths and change

void QCoreApplicationPrivate::prependApplicationPathToLibraryPaths()
{
#ifndef QT_NO_LIBRARY
    QStringList *app_libpaths = coreappdata()->app_libpaths;
    if (!app_libpaths)
        coreappdata()->app_libpaths = app_libpaths = new QStringList;
    QString app_location = QCoreApplication::applicationFilePath();
    app_location.truncate(app_location.lastIndexOf(QLatin1Char('/')));
#ifdef Q_OS_WINRT
    if (app_location.isEmpty())
        app_location.append(QLatin1Char('/'));
#endif
    app_location = QDir(app_location).canonicalPath();
    if (QFile::exists(app_location) && !app_libpaths->contains(app_location))
        //CHANGE THIS ROW: app_libpaths->append(app_location);
        //TO FOLLOWING
        app_libpaths->prepend(app_location);
#endif
}

InApp solution

Unfortunately it isn’t possible to simply change this behavior from your app. All of these operations happen directly in QCoreApplication constructor so if you try to change it after, it’s too late.

The temporary solution before this problem will be resolved is to reinitialize library paths before QCoreApplication is initialized. It’s necessary to clean libray paths, compute new paths and re-initialize QCoreApplication::libraryPaths before QCoreApplication object is initialized. This can be done in main.cpp of your application before you will create QApplication/QCoreApplication object.

  QString executable = argv[0];
  QString executablePath = executable.mid(0,executable.lastIndexOf("\\"));
  QString installPathPlugins = QLibraryInfo::location(QLibraryInfo::PluginsPath);
  QCoreApplication::removeLibraryPath(installPathPlugins);
  QCoreApplication::addLibraryPath(installPathPlugins);
  QCoreApplication::addLibraryPath(executablePath);

It’s not a nice solution, but it works. I tried  to report this issue also to bugreports.QtProject, so maybe in later version this will be fixed.

How to create .pdb files also for Release version of Qt library

For debugging purposes it’s a good idea to keep .pdb files also for release version of your libraries. Unfortunately in default configuration Qt library doesn’t generate .pdb files. It’s a not big deal, create .pdb files for release version it’s pretty simple.

1) Add compilation flags to .pro file

The easiest way how to create pdb files is pass /Zi switch to compiler and /Debug to linker.

QMAKE_CXXFLAGS+=/Zi
QMAKE_LFLAGS+= /INCREMENTAL:NO /Debug

2) Solution via mkspec

As pointed out in this QtProject article, another way is to edit mkspec so any future compilations will have this option turned on. It’s necessary to modify correct mkspec file, for example

Qt</span>\mkspecs\win32-msvc2005\qmake.conf

and add following lines (it should be sufficient to add only /Zi and /Debug switch):

QMAKE_CFLAGS_RELEASE += /Zi
QMAKE_LFLAGS_RELEASE += /INCREMENTAL:NO /DEBUG

Qt app crash on mac when executed with parameters

Our reporting tool occasional crashed on MacOs when was executed with several command line params. Command looked like this:

/app-path/OrmDesigner2 -crash-report -dump-directory /var/folders/4z/k1r35jnn4v77mzl7hzf8wvvw0000gn/T/OrmDesigner2/CrashReports

When inspecting exception report, we found following stack trace:

Thread 0 Crashed:  Dispatch queue: com.apple.main-thread
0   QtCore                         0x0000000100d671da QString::fromLocal8Bit(char const*, int) + 42
1   QtCore                         0x0000000100e72dac QCoreApplication::arguments() + 124
2   QtGui                          0x00000001000de8ef -[QCocoaApplicationDelegate application:openFiles:] + 223
3   com.apple.AppKit               0x00007fff805f3b52 -[NSApplication(NSAppleEventHandling) _handleAEOpenDocumentsForURLs:] + 505
4   com.apple.AppKit               0x00007fff804c0065 -[NSApplication(NSAppleEventHandling) _handleCoreEvent:withReplyEvent:] + 217
5   com.apple.Foundation           0x00007fff800f90d6 -[NSAppleEventManager dispatchRawAppleEvent:withRawReply:handlerRefCon:] + 360
6   com.apple.Foundation           0x00007fff800f8f06 _NSAppleEventManagerGenericHandler + 114
7   com.apple.AE                   0x00007fff82b1c32b aeDispatchAppleEvent(AEDesc const*, AEDesc*, unsigned int, unsigned char*) + 162
8   com.apple.AE                   0x00007fff82b1c224 dispatchEventAndSendReply(AEDesc const*, AEDesc*) + 32
9   com.apple.AE                   0x00007fff82b1c12b aeProcessAppleEvent + 210
10  com.apple.HIToolbox            0x00007fff875ed619 AEProcessAppleEvent + 48
11  com.apple.AppKit               0x00007fff803c5095 _DPSNextEvent + 1191
12  com.apple.AppKit               0x00007fff803c4801 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 155
13  com.apple.AppKit               0x00007fff8038a68f -[NSApplication run] + 395
14  QtGui                          0x00000001000e7c04 QEventDispatcherMac::processEvents(QFlags) + 1588
15  QtCore                         0x0000000100e70774 QEventLoop::processEvents(QFlags) + 68
16  QtCore                         0x0000000100e70a94 QEventLoop::exec(QFlags) + 180
17  QtCore                         0x0000000100e720bc QCoreApplication::exec() + 188
18  com.inventic.ormdesigner       0x0000000100003447 main + 46 (main.cpp:31)
19  com.inventic.ormdesigner       0x00000001000032e0 start + 52

After some google-searching I found out that the problem is in core of our application. When subclassing Qt QApplication, it’s necessary to pass argc with reference, not by value. So after adding one little &, everything works like charm!

//construct application object
CApplication(int& argc, char **argv);

External links

VS2010 always rebuilding moc files on Qt

My VS2010 on every build always recreate all .moc files and associated .cpp files. Problem was in incredibuild clausule during compilation of my Qt libraries.

 

      <AdditionalInputs Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">Rem;axApplication\axAppException\dialogUnhandledException.ui;%(AdditionalInputs)</AdditionalInputs>
      <Command Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">Rem IncrediBuild_AllowRemote &#x000D;&#x000A;if errorlevel 1 goto VCEnd&#x000D;&#x000A; Rem IncrediBuild_OutputFile ..\GeneratedFiles\ui_dialogUnhandledException.h &#x000D;&#x000A;if errorlevel 1 goto VCEnd&#x000D;&#x000A; Q:\ExternalLibraries\qt-git-32bit\bin\uic.exe axApplication\axAppException\dialogUnhandledException.ui -o ..\GeneratedFiles\ui_dialogUnhandledException.h</Command>
      <Message Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">UIC axApplication\axAppException\dialogUnhandledException.ui</Message>
      <Outputs Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">..\GeneratedFiles\ui_dialogUnhandledException.h;%(Outputs)</Outputs>

instead of

      <AdditionalInputs Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">Q:\ExternalLibraries\qt-4.8.2-32bit\bin\uic.exe;axApplication\axAppException\dialogUnhandledException.ui;%(AdditionalInputs)</AdditionalInputs>
      <Command Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">Q:\ExternalLibraries\qt-4.8.2-32bit\bin\uic.exe axApplication\axAppException\dialogUnhandledException.ui -o ..\GeneratedFiles\ui_dialogUnhandledException.h</Command>
      <Message Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">UIC axApplication\axAppException\dialogUnhandledException.ui</Message>
      <Outputs Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">..\GeneratedFiles\ui_dialogUnhandledException.h;%(Outputs)</Outputs>

Although qmake added all these incredibuild defines to my VS project, Incredibuild still doesn’t compile it in parallel so it was totally useless. The most simply way is edit incredibuild_xge.prf and turn-off it.

mkspecs\features\incredibuild_xge.prf 

contains(TEMPLATE, "XXXvc.*") {

How to enable VS2010 logging

Qt on OSX Maverick – Undefined symbols for architecture x86_64

The problem is that starting with OSX 10.9 Apple changed default standard c++ library from libstdc++ to libc++.

Qt binary distribution compile with -stdlib=libstdc++ to be compatible with 10.6, Xcode 5 on 10.9 will select -stdlib=libc++ by default (for OS X 10.7 and better only). So symbol using classes from the standard library (like std::string in this case) will not resolve correctly at link time.

Undefined symbols for architecture x86_64:
"boost::filesystem::path_traits::convert(char const*, char const*, std::basic_string&lt;wchar_t, std::char_traits&lt;wchar_t&gt;, std::allocator&lt;wchar_t&gt; &gt;&amp;, std::codecvt&lt;wchar_t, char, __mbstate_t&gt; const&amp;)", referenced from:
boost::filesystem::path::wstring(std::codecvt&lt;wchar_t, char, __mbstate_t&gt; const&amp;) const in filePath.o
boost::filesystem::path::wstring(std::codecvt&lt;wchar_t, char, __mbstate_t&gt; const&amp;) const in fileEnumerator.o
boost::filesystem::path::wstring(std::codecvt&lt;wchar_t, char, __mbstate_t&gt; const&amp;) const in directoryHelper.test.o

So it’s necessary to compile all libraries with one type of libstdc++ (or libc++). Because I need to keep 10.6 compatibility, it’s necessary to compile boost and other libraries with libstdc++ dependency.

To check which library is used, use otool tool:

otool -L library.dylib

As result you will get something like this (check /libc++1.dylib):

otool -L boost/lib/libboost_filesystem.dylib
boost/lib/libboost_filesystem.dylib:
	boost/lib/libboost_filesystem.dylib (compatibility version 0.0.0, current version 0.0.0)
	boost/lib/libboost_system.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1)

How to fix it for boost

it’s necessary to recompile boost (and don’t forget to remove ./bin.v2 directory) with these params:

  ./b2 cxxflags="-stdlib=libstdc++" linkflags="-stdlib=libstdc++" ...

and run otool again:

boost/lib/libboost_filesystem.dylib:
	boost/lib/libboost_filesystem.dylib (compatibility version 0.0.0, current version 0.0.0)
	boost/lib/libboost_system.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 60.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1)

How to fix it for CMake libraries

In case you’re using library which is built by CMake system, you  need to add following flag:

cmake -DCMAKE_CXX_FLAGS="-stdlib=libstdc++"

How to fix other libraries

For any other library it’s necessary to pass libstdc++ flag in any available way, for example modify makefile and add :

CXXFLAGS = -stdlib=libstdc++

Second way how to fix it, compile app using latest compiler

Another way is to update mkspecs to compile for latest MacOS version, after that, compiler will be the same like compiler used on all other libraries. To do that, it’s necessary to update file:

/usr/local/Qt-5.3.1/mkspecs/macx-clang/qmake.conf

and change following line:

QMAKE_MACOSX_DEPLOYMENT_TARGET = 10.6

to latest 10.9 version:

QMAKE_MACOSX_DEPLOYMENT_TARGET = 10.9

More info about this process check here:

http://stackoverflow.com/questions/20342896/solved-qt5-1-qt5-2-mac-os-10-9-mavericks-xcode-5-0-2-undefined-symbols

And that’s all

Hope this post saves you a lot of time I have to spent by searching these answers 😉

External links: