Notarized Qt binaries for OSX

Notarization sh script for automate whole process. Script is created based on several examples found on the web. Links to these examples can be found bellow (mostly thanks to Logcg.com blog).

Notarize DMG file

#!/bin/sh

APPLICATION_PATH=$1
UNDLE_ID="___BUNDLE_ID____"
APPLE_USER="___APPLE_USER____"
APPLE_PASSWORD="___APLE_APPSPECIFIC_PASSWORD___"

 
echo "Running notarize-app command for file $APPLICATION_PATH"
xcrun altool --notarize-app -t osx -f "$APPLICATION_PATH" --primary-bundle-id=$BUNDLE_ID -u $APPLE_USER -p $APPLE_PASSWORD &> notarize-status.txt

echo "Notarize-app complete. Result: "
cat notarize-status.txt

uuid=`cat notarize-status.txt | grep -Eo '\w{8}-(\w{4}-){3}\w{12}$'`
echo "Request UUID is $uuid"

while true; do
    echo "checking for notarization..."
    xcrun altool --notarization-info "$uuid" --username $APPLE_USER --password $APPLE_PASSWORD &> notarize-response.txt
    
    echo "Response:"
    cat notarize-response.txt
    
    t=`cat notarize-response.txt | grep "success"`
    f=`cat notarize-response.txt | grep "invalid"`
    if [[ "$t" != "" ]]; then
        echo "notarization done! Stampling application $APPLICATION_PATH"
        xcrun stapler staple "$APPLICATION_PATH"
        echo "stapler done!"
        break
    fi
    if [[ "$f" != "" ]]; then
        echo "$r"
        return 1
    fi
    echo "not finish yet, sleep 30sec then check again..."
    sleep 30
done

Notarize .app / .zip

#!/bin/sh

APPLICATION_PATH=$1
ZIP_PATH=./application-to-notarize.zip
BUNDLE_ID="___BUNDLE_ID____"
APPLE_USER="___APPLE_USER____"
APPLE_PASSWORD="___APLE_APPSPECIFIC_PASSWORD___"


echo "Packing app $APPLICATION_PATH to zip file $ZIP_PATH"
ditto -ck --rsrc --sequesterRsrc $APPLICATION_PATH $ZIP_PATH
 
echo "Running notarize-app command for file $ZIP_PATH"
xcrun altool --notarize-app -t osx -f "$ZIP_PATH" --primary-bundle-id=$BUNDLE_ID -u $APPLE_USER -p $APPLE_PASSWORD &> notarize-status.txt

echo "Notarize-app complete. Result: "
cat notarize-status.txt

uuid=`cat notarize-status.txt | grep -Eo '\w{8}-(\w{4}-){3}\w{12}$'`
echo "Request UUID is $uuid"

while true; do
    echo "checking for notarization..."
    xcrun altool --notarization-info "$uuid" --username $APPLE_USER --password $APPLE_PASSWORD &> notarize-response.txt
    
    echo "Response:"
    cat notarize-response.txt
    
    t=`cat notarize-response.txt | grep "success"`
    f=`cat notarize-response.txt | grep "invalid"`
    if [[ "$t" != "" ]]; then
        echo "notarization done! Stampling application $APPLICATION_PATH"
        xcrun stapler staple "$APPLICATION_PATH"
        echo "stapler done!"
        break
    fi
    if [[ "$f" != "" ]]; then
        echo "$r"
        return 1
    fi
    echo "not finish yet, sleep 30sec then check again..."
    sleep 30
done

Useful links:

How to generate new Apple developer certificate

“KeyChain Access application” -> KeyChain access -> Certificate assistant -> Request certificate from authority

  • Developer application -> Next -> Next
  • Upoad request file, download generated certificate
  • Doubleclick on certificate and import it to Keychain access

Usage

For the first time it’s necessary to use codesign from the OSX directly, not via putty/remote.

Execute

codesign --deep --force --verify --sign "$SIGNNAME" --options "runtime" $APP

and enter keychains passwords and confirm with “Always use” . Next time you can use putty as usual.

 

Crash in QTreeWidget / QTreeView index mapping on Mac OSX 10.10 part III

So, one more attempt. Previous articles (part1, part2) mentioned possible solutions to fix crash insinde the QTreeWidget and QAccessibleTableCell.

Unfortunately, deselecting current item still doesn’t fix all issues. The problem with deselection is that it’s not handled correctly via QAccessibleTableCell:

If current index isn’t valid, QAccessible doesn’t correctly update currect QAccessibleTableCell object which caused all this evil crashes.

So there are two ways how to fix it. One is to manually set QAccessibleTableCell but I didn’t find a way how to do that. The second way is to disable QAccessible for Property editor. Unfortunately it’s not an easy task. In QAccessible exists method setActive:

QAccessible::setActive(false)

But this method works only with AccessibleObserver class but not with QPlatformAcessibility which handles real QAcessible::isActive result. Because of this it’s necessary to update setActive method in file qacessible.cpp inside the Qt.

Original method:


void QAccessible::setActive(bool active)
{
  for (int i = 0; i < qAccessibleActivationObservers()->count() ;++i)
    qAccessibleActivationObservers()->at(i)->accessibilityActiveChanged(active);
}

and updated version:

void QAccessible::setActive(bool active)
{
  #ifndef QT_NO_ACCESSIBILITY
    if ( QPlatformAccessibility *pfAccessibility = platformAccessibility() )
      pfAccessibility->setActive(active);
  #endif

  for (int i = 0; i < qAccessibleActivationObservers()->count() ;++i)
    qAccessibleActivationObservers()->at(i)->accessibilityActiveChanged(active);
}

In case you will find better way how to turn off accessibility, please let me know.

But there is still one more problem. Occasionally OS X decides to switch accessible back on:

And this is probably the reason why everything works ok on older OS X but not on 10.10. Because OS X 10.10 randomly turns accessible on which causes this crash. Accessible is usually turned on when opening some modal dialog.

This explains why opening an empty dialog increased the probability to crash the application. The only solution I have found is to disable QAccessible every time the PropertyEditor is cleared up.

Crash in QTreeWidget / QTreeView index mapping on Mac OSX 10.10 part II

As mentioned in the first part of this article, the workaround with focus unfortunately didn’t work. In some cases application is still crashing. So after next hours of debugging and unsuccessful consulting the problem on stack overflow I probably found a the core reason of this bug and the solution too.

2015-05-18_0850

 

The core of this problem doesn’t lie in the QTreeModel, but in Qt implementation of Cocoa Accessible. The problem is that although the PropertyTree is correctly cleared, there is object QAccessibleTableCell which isn’t correctly updated after

clear()

.

This means that although the PropertyTree and TreeWidget is empty, the QAccessibleTableCell object still holds m_index variable to invalid element. And when

 NSAcessibleEntryPointrAttributeNames 

is executed, m_index is translated to QTreeWidget element pointer and accessed. And this means that application completely crashes.

The solution is pretty simple. Together with property editor clear set also current index and selected index to NULL.

	propertyBrowser->clear();
	propertyManagers->clear();
	propertyManagers->treeWidget()->setCurrentItem(NULL);
	propertyManagers->treeWidget()->setItemSelected(NULL, false);

And that is it. Thanks for this your application will not crash any more (on this bug 😉 ).

 

OS X codesign failed: bundle format is ambiguous (could be app or framework)

This error can be caused by many things but I know about one more which I didn’t find anywhere else ;-).

In case you compile and deploy your app by using qtmacdeploy and sign your application immediately, everything will probably works fine. The problem occurs, when you need to copy your application to different location (for example during dmg building). In such cases, this error can occur:

bundle format is ambiguous (could be app or framework)

Althought codesign is executed as always, singing isn’t successful:

codesign --deep --force --verbose --sign "$SIGNNAME" ./Skipper.app
/path/Skipper.app/Contents/Frameworks/QtCore.framework: bundle format is ambiguous (could be app or framework)

The problem is, that during the copy it’s necessary to keep all symbolic links inside frameworks. Without this, singing will fail.

So instead of

cp -r ./Source ./Destination

it’s necessary to use

cp -R ./Source ./Destination

Mac OS – app can’t be opened because the identity of the developer cannot be confirmed.

Starting with OS X 10.10 existing code signing method doesn’t work. If you have application signed for 10.9 and application works without problems, with 10.10 you will get following error:

2015-02-05_1003

How to verify application sign status from command line:

codesign -dvvv /Applications/APP.app

Executable=/Applications/Skipper.app/Contents/MacOS/Skipper
Identifier=com.skipper.Skipper
Format=bundle with Mach-O thin (x86_64)
CodeDirectory v=20100 size=239848 flags=0x0(none) hashes=11986+3 location=embedded
Hash type=sha1 size=20
CDHash=98839e7aa72de4105ac5ad8a2612682ba3bca53f
Signature size=4237
Authority=Developer ID Application: Inventic s.r.o. (6BYV46LH6T)
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Signed Time=03 Feb 2015 17:38:21
Info.plist entries=10
TeamIdentifier=not set
Sealed Resources version=1 rules=4 files=44
Internal requirements count=1 size=300

As it’s seems from the verification output, application is correctly signed but OSX doesn’t accept it. Another way how to verify application sign status is via spctl command:

spctl --assess --type execute --verbose Skipper.app/

Skipper.app/: rejected
source=obsolete resource envelope

We have some error at least. Now it’s necessary to find out what is wrong. We can try one more test:

codesign -v Skipper.app/
Skipper.app/: resource envelope is obsolete (version 1 signature)

where we dest little bit more details. All these errors we get only on 10.10 mac, not on 10.9 or older.

After another investigation I found following article. The most important part is:

“Important: For your apps to run on updated versions of OSX they must be signed on OS X version 10.9 or later and thus have a version 2 signature.”

Another post about this topic is in felix-schwarz.org blog.

So ,it’s bad. We need to update our build machine to 10.9 or at least create new “sign machine” and make sure that everything will work as expected.

Additional links

Qt5 application hangs-up when QNetworkAccessManager and QEventLoop is used on Mac OS

To be more specific, problem occurs only in very specific circumstances. It’s in situation, when application is compiled as console-app and it’s compiled on Mac OS X version older than 10.9:

CONFIG -= gui
CONFIG -= console

and when you’re using QEventLoop::exec in mode processing all events except user events (QEventLoop::ExcludeUserInputEvents):


//it's only demonstration, not full code...
QNetworkAccessManager manager; ...
QNetworkRequest request; ...
QNetworkReply *reply = manager->post(request,arData);
...

eventLoop.exec(QEventLoop::ExcludeUserInputEvents);

In this situation, application hangs-up. When the application is compiled with GUI mode or when application is compiled on Linux/Windows (no matter if gui or console), everything works find. To fix this problem, it’s necessary to implement hack similar to this (simplified version):

   bool bAllowUserInputEvents = false;

#if defined(PLATFORM_MACOS) && defined(AX_APP_CONSOLE)
   bAllowUserInputEvents = true;
#endif

  if ( bAllowUserInputEvents == false )
    eventLoop.exec(QEventLoop::ExcludeUserInputEvents);
  else
    eventLoop.exec(QEventLoop::AllEvents);

How to change plist value on MacOS 10.9+

When .plist file is manually updated on MacOS 10.9+, changes are not directly applied to plist cache. This means that  these changes aren’t visible to executed application (even if you run it after plist change).

It’s necessary to tell plist change to reload this settings. There are two ways how to do it. First one is simpler but doesn’t work if you completely remove .plist file:

defaults read ~/Library/Preferences/com.xxx.xxx.xxx

Second method is much more efficient, but requires to completely kill plist cache. This cache is immediately executed again and during this launch all plist files are scanned and loaded again:

kill cfprefsd

Qt app crash on mac when executed with parameters

Our reporting tool occasional crashed on MacOs when was executed with several command line params. Command looked like this:

/app-path/OrmDesigner2 -crash-report -dump-directory /var/folders/4z/k1r35jnn4v77mzl7hzf8wvvw0000gn/T/OrmDesigner2/CrashReports

When inspecting exception report, we found following stack trace:

Thread 0 Crashed:  Dispatch queue: com.apple.main-thread
0   QtCore                         0x0000000100d671da QString::fromLocal8Bit(char const*, int) + 42
1   QtCore                         0x0000000100e72dac QCoreApplication::arguments() + 124
2   QtGui                          0x00000001000de8ef -[QCocoaApplicationDelegate application:openFiles:] + 223
3   com.apple.AppKit               0x00007fff805f3b52 -[NSApplication(NSAppleEventHandling) _handleAEOpenDocumentsForURLs:] + 505
4   com.apple.AppKit               0x00007fff804c0065 -[NSApplication(NSAppleEventHandling) _handleCoreEvent:withReplyEvent:] + 217
5   com.apple.Foundation           0x00007fff800f90d6 -[NSAppleEventManager dispatchRawAppleEvent:withRawReply:handlerRefCon:] + 360
6   com.apple.Foundation           0x00007fff800f8f06 _NSAppleEventManagerGenericHandler + 114
7   com.apple.AE                   0x00007fff82b1c32b aeDispatchAppleEvent(AEDesc const*, AEDesc*, unsigned int, unsigned char*) + 162
8   com.apple.AE                   0x00007fff82b1c224 dispatchEventAndSendReply(AEDesc const*, AEDesc*) + 32
9   com.apple.AE                   0x00007fff82b1c12b aeProcessAppleEvent + 210
10  com.apple.HIToolbox            0x00007fff875ed619 AEProcessAppleEvent + 48
11  com.apple.AppKit               0x00007fff803c5095 _DPSNextEvent + 1191
12  com.apple.AppKit               0x00007fff803c4801 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 155
13  com.apple.AppKit               0x00007fff8038a68f -[NSApplication run] + 395
14  QtGui                          0x00000001000e7c04 QEventDispatcherMac::processEvents(QFlags) + 1588
15  QtCore                         0x0000000100e70774 QEventLoop::processEvents(QFlags) + 68
16  QtCore                         0x0000000100e70a94 QEventLoop::exec(QFlags) + 180
17  QtCore                         0x0000000100e720bc QCoreApplication::exec() + 188
18  com.inventic.ormdesigner       0x0000000100003447 main + 46 (main.cpp:31)
19  com.inventic.ormdesigner       0x00000001000032e0 start + 52

After some google-searching I found out that the problem is in core of our application. When subclassing Qt QApplication, it’s necessary to pass argc with reference, not by value. So after adding one little &, everything works like charm!

//construct application object
CApplication(int& argc, char **argv);

External links

Using ssh as proxy/tunnel between Mac OS and Linux

How to connect to remote computer with private key by using ssh:

#register private key
ssh-add ~/dev/Private/ludek_vodicka_dsa.openssh

#connect to computer and forward local port to remote computer and remote port
ssh my-computer.com -p PORT -lUSER -L LOCAL_PORT:REMOTE_COMPUTER:REMOTE_PORT

#connect to computer and create SOCKS proxy on port SOCKS_PORT
ssh my-computer.com -p PORT -lUSER -D SOCKS_PORT

Note:

If your private key was created for Windows, it will probably not work on linux/mac. It’s necessary to convert it by using puttygen (on mac or windows). You need to open it by puttygen and choose “Save as openssh” from menu.

Additional resources

Qt on OSX Maverick – Undefined symbols for architecture x86_64

The problem is that starting with OSX 10.9 Apple changed default standard c++ library from libstdc++ to libc++.

Qt binary distribution compile with -stdlib=libstdc++ to be compatible with 10.6, Xcode 5 on 10.9 will select -stdlib=libc++ by default (for OS X 10.7 and better only). So symbol using classes from the standard library (like std::string in this case) will not resolve correctly at link time.

Undefined symbols for architecture x86_64:
"boost::filesystem::path_traits::convert(char const*, char const*, std::basic_string&lt;wchar_t, std::char_traits&lt;wchar_t&gt;, std::allocator&lt;wchar_t&gt; &gt;&amp;, std::codecvt&lt;wchar_t, char, __mbstate_t&gt; const&amp;)", referenced from:
boost::filesystem::path::wstring(std::codecvt&lt;wchar_t, char, __mbstate_t&gt; const&amp;) const in filePath.o
boost::filesystem::path::wstring(std::codecvt&lt;wchar_t, char, __mbstate_t&gt; const&amp;) const in fileEnumerator.o
boost::filesystem::path::wstring(std::codecvt&lt;wchar_t, char, __mbstate_t&gt; const&amp;) const in directoryHelper.test.o

So it’s necessary to compile all libraries with one type of libstdc++ (or libc++). Because I need to keep 10.6 compatibility, it’s necessary to compile boost and other libraries with libstdc++ dependency.

To check which library is used, use otool tool:

otool -L library.dylib

As result you will get something like this (check /libc++1.dylib):

otool -L boost/lib/libboost_filesystem.dylib
boost/lib/libboost_filesystem.dylib:
	boost/lib/libboost_filesystem.dylib (compatibility version 0.0.0, current version 0.0.0)
	boost/lib/libboost_system.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1)

How to fix it for boost

it’s necessary to recompile boost (and don’t forget to remove ./bin.v2 directory) with these params:

  ./b2 cxxflags="-stdlib=libstdc++" linkflags="-stdlib=libstdc++" ...

and run otool again:

boost/lib/libboost_filesystem.dylib:
	boost/lib/libboost_filesystem.dylib (compatibility version 0.0.0, current version 0.0.0)
	boost/lib/libboost_system.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 60.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1)

How to fix it for CMake libraries

In case you’re using library which is built by CMake system, you  need to add following flag:

cmake -DCMAKE_CXX_FLAGS="-stdlib=libstdc++"

How to fix other libraries

For any other library it’s necessary to pass libstdc++ flag in any available way, for example modify makefile and add :

CXXFLAGS = -stdlib=libstdc++

Second way how to fix it, compile app using latest compiler

Another way is to update mkspecs to compile for latest MacOS version, after that, compiler will be the same like compiler used on all other libraries. To do that, it’s necessary to update file:

/usr/local/Qt-5.3.1/mkspecs/macx-clang/qmake.conf

and change following line:

QMAKE_MACOSX_DEPLOYMENT_TARGET = 10.6

to latest 10.9 version:

QMAKE_MACOSX_DEPLOYMENT_TARGET = 10.9

More info about this process check here:

http://stackoverflow.com/questions/20342896/solved-qt5-1-qt5-2-mac-os-10-9-mavericks-xcode-5-0-2-undefined-symbols

And that’s all

Hope this post saves you a lot of time I have to spent by searching these answers 😉

External links:

Useful Mac OS tools for developer

System Tools

External articles:

Programming

  • SmartSVN (free for personal use,  $69/license) – svn gui tool
  • SmartGit/Hg  (free) – git / hg gui tool

File Manager