Mobile development

Embedding Python in Windows Mobile C++

2009-04-25T00:24:00.000-07:00

Build

1. Download and unpack Python sources PythonCE-2.5-20061219-source.zip from www.sourceforge.net

2. On www.sourceforge.net in Tracker>Patches section click on "PythonCE 2.5 for WM Smartphone" are information about the patch for WM5/WM6 OS version, which can be build with Vs2005

3. Download the patch file (Python-2.5-20071004-patch.zip) from link http://sourceforge.net/project/showfiles.php?group_id=104228&package_id=247631

4. Unpack the patch and

- copy "PC" folder to original Python sources

- use GNU patch utility (http://gnuwin32.sourceforge.net/packages/patch.htm) to both .patch files

- patch -p1 << smartphone.patch

- patch -p1 << vs2005-compiler.patch

5. Go to directory ...\PCbuild\WinCE\ and start the build command "scons.bat"

6. For debug builds use "scons.bat debug-all"

Use

1. Copy python25.dll to the mobile

2. Link against the python25.lib

or for debug

1. Copy python25_d.dll to the mobile

2. Link against the python25_d.lib

Problem

The only problem is the DLL size, which is over 2.5 megabytes, I have asked here what I can do with it:

https://sourceforge.net/forum/forum.php?thread_id=3241586&forum_id=358833

Links

http://sourceforge.net/projects/pythonce/

Embedding Python into Symbian C++

2009-04-23T22:56:00.000-07:00

Introduction

Use of the Python scripting language is an excellent way how to extend the functionality of the C++ program. It is ideal especially for the game scripting or for complex configuration files. The Python itself can be downloaded from the www.sourceforge.net pages. For the embedding on the S60 3^rd platform the sis file (PythonForS60_1_4_5_3rdEd.sis) is necessary to be installed on the phone. In fact the only necessary component for the embedding is the Python interpreter DLL library python222.dll included within the installation sis file. If embedding the sis file into application sis, which invokes the Python from C++, is not the suitable way or capabilities of the DLL must be changed or there is simply some other reason to not use the installation directly, we have the possibility to download sources (pys60-1.4.5_src.zip) and build the python interpreter DLL on our own. With the help of description in the readme.txt from the sources zip file it is quite easy. If the own build of the interpreter DLL will be part of the sis installation package you have to remember to change its UID and name to not clash with the original Python installation.

Portability

The advantage of the Python is also its availability on Windows Mobile platform, so if you are writing the portable C++ application it is currently the only choice (as far as I know) for the scripting language as other good languages like Lua or GameMonkey are not ported to both platforms.

How to embed

For embedding scripts into C++ code there are two possibilities, either use directly the Python/C API or use the CSPyInterpreter Symbian wrapper class. While use of the wrapper class is easier to use, it does not allow handling input/output script (module) parameters, calling selected functions and getting back the result.

To be able to use the Python in C++ code the SDK from the www.sourceforge.net pages has to be downloaded and installed. For each S60 3^rd platform edition there is the separate package (e.g. PythonForS60_1_4_5_SDK_3rdEd.zip) – it contains include header files and the Python222.lib export library we need to link against.

Using the CSPyInterpreter class is pretty simple, as shown in the code snippet below:

...

RunPythonSimpleScriptL( _L("c:\\resource\\writeToFile.py") );

...

void RunPythonSimpleScriptL( const TDesC& aScriptName )

{

// Create a Python interpreter

CSPyInterpreter* it = CSPyInterpreter::NewInterpreterL();

CleanupStack::PushL( it );

// Convert the script name to char*

HBufC8* scriptName = CnvUtfConverter::ConvertFromUnicodeToUtf8L( aScriptName );

CleanupStack::PushL( scriptName );

char* scriptNameChar = (char*)scriptName->Des().PtrZ();

// Run script

TInt err = it->RunScript( 1, &scriptNameChar );

User::LeaveIfError( err );

// Clean everything

CleanupStack::PopAndDestroy( scriptName );

CleanupStack::PopAndDestroy( it );

}

Python script writeToFile.py:

file = open("c:\\test.txt", 'w')

file.write("This is the new content of test.txt :-)")

file.close()

In a more complex example we need:

1. Convert data values from Symbian C++ to native C representation and then to Python representation

2. Perform a function call from a Python script using converted values

3. Convert data values from the Python back to the Symbian C++

Following code snippets do the stuff. First some initialization is done; the c:\scripts path is specified as a script repository. If the path is not specified .py scripts are expected to be saved in the c:\resource directory. Then all parameters are converted to the PyObject type and the script is loaded by calling the PyImport_Import() method. Later on the function object is retrieved from the script and is called. The result is converted from PyObject to char* and then to TPtrC8.

void RunPythonScript1L()

{

// Create a Python interpreter

CSPyInterpreter* it = CSPyInterpreter::NewInterpreterL();

CleanupStack::PushL( it );

// Save state of any current Python interpreter, and acquire the

// interpreter lock

PyEval_RestoreThread( PYTHON_TLS->thread_state );

// Set path for .py scripts

_LIT8(KPyExecPath, "c:\\scripts\0");

PySys_SetPath((char*)TPtrC8(KPyExecPath).Ptr());

// .py module name

char *module_name = "simpleScript";

// .py module function name

char *function_name = "returnString";

// Python API objects

PyObject *py_module_name, *py_module, *py_function, *py_result;

// Module (script) name (Return a new string object with a copy of the string)

py_module_name = PyString_FromString( module_name );

// The function imports the module name, potentially using the given

// globals and locals to determine how to interpret the name in a package context

py_module = PyImport_Import( py_module_name );

Py_DECREF( py_module_name );

if ( py_module )

{

// Retrieve an attribute named from object (i.e. function from module)

py_function = PyObject_GetAttrString( py_module, function_name );

if ( py_function )

{

// Call a callable Python object

py_result = PyObject_CallObject( py_function, NULL );

Py_DECREF( py_module );

Py_DECREF( py_function );

// Convert result from PyObject to char*

int result_len = -1;

// .py function result

char *result = NULL;

result = PyString_AsString( py_result ) ;

result_len = strlen( result ) ;

// Make a Symbian descriptor pointer to the char * response

TPtrC8 sym_result( (TUint8*)result, result_len );

if ( sym_result.Compare( _L8("Hello world") ) != 0 )

{

// The result is other then expected

Py_DECREF( py_result );

PyEval_SaveThread();

User::Leave( KErrGeneral );

}

Py_DECREF( py_result );

}

else

{

// Function not found in the module

Py_DECREF( py_module );

PyEval_SaveThread();

User::Leave( KErrNotFound );

}

else

{

// Module not found

PyEval_SaveThread();

User::Leave( KErrNotFound );

}

PyEval_SaveThread();

CleanupStack::PopAndDestroy( it );

}

The second example shows the situation, where the method to sum two number is called, so we need to pass to Python script two integer values we want to be added. This is done by PyTuple_XXX() methods, which construct kind of dynamic array into which we consequently insert PyObject arguments for the script function.

void RunPythonScript2L()

{

// Create a Python interpreter

CSPyInterpreter* it = CSPyInterpreter::NewInterpreterL();

CleanupStack::PushL( it );

// Save state of any current Python interpreter, and acquire the

// interpreter lock

PyEval_RestoreThread( PYTHON_TLS->thread_state );

// Set path for .py scripts

_LIT8(KPyExecPath, "c:\\scripts\0");

PySys_SetPath((char*)TPtrC8(KPyExecPath).Ptr());

// .py module name

char *module_name = "simpleScript";

// .py module function name

char *function_name = "sum";

// .py function result

int result = -1;

// Python API objects

PyObject *py_module_name, *py_module, *py_function, *py_result;

// Python funtcion agum,ents

PyObject *py_arguments, *py_argument;

// Module (script) name (Return a new string object with a copy of the string)

py_module_name = PyString_FromString( module_name );

// The function imports the module name, potentially using the given

// globals and locals to determine how to interpret the name in a package context

py_module = PyImport_Import( py_module_name );

Py_DECREF( py_module_name );

if ( py_module )

{

// Retrieve an attribute named from object (i.e. function from module)

py_function = PyObject_GetAttrString( py_module, function_name );

if ( py_function )

{

// Create fumction arguments object of size 2

py_arguments = PyTuple_New( 2 );

// 1st argument

py_argument = PyInt_FromLong(5);

PyTuple_SetItem(py_arguments, 0, py_argument);

// 2nd argument

py_argument = PyInt_FromLong(3);

PyTuple_SetItem(py_arguments, 1, py_argument);

// Call a callable Python object

py_result = PyObject_CallObject( py_function, py_arguments );

Py_DECREF( py_arguments );

Py_DECREF( py_module );

Py_DECREF( py_function );

// Convert result from PyObject to char*

result = PyInt_AsLong( py_result ) ;

if ( result != 8 )

{

// The result is other then expected

Py_DECREF( py_result );

PyEval_SaveThread();

User::Leave( KErrGeneral );

}

Py_DECREF( py_result );

}

else

{

// Function not found in the module

Py_DECREF( py_module );

PyEval_SaveThread();

User::Leave( KErrNotFound );

}

else

{

// Module not found

PyEval_SaveThread();

User::Leave( KErrNotFound );

}

PyEval_SaveThread();

CleanupStack::PopAndDestroy( it );

}

The simpleScript.py:

# Returns the string simply

def returnString():

return "Hello world"

# Returns sum of 2 given numbers

def sum(a,b):

c = a + b

return c

Links

S60 Python home page:

http://www.forum.nokia.com/Resources_and_Information/Tools/Runtimes/Python_for_S60/

Sourceforge:

http://sourceforge.net/project/showfiles.php?group_id=154155&package_id=171153

Python/C API

http://docs.python.org/extending/embedding.html

Receiving/loosing the focus (WM_ACTIVATE) on Windows Mobile

2009-02-23T23:19:00.000-08:00

When DirectX application is running in full screen (DDSCL_FULLSCREEN) it does not receive any focus changed events, namely WM_ACTIVATE message.

As stated in http://msdn.microsoft.com/en-us/library/ms847103.aspx:
"...The exclusive (full-screen) mode prevents other applications from allocating some surface types and from drawing to the primary display. The exclusive mode also prevents other windows on the system from coming to the foreground, so DirectDraw applications must watch for system events (such as incoming phone calls) so they can relinquish control back to the system when necessary..."

The possible solution for that is discussed here http://social.msdn.microsoft.com/Forums/en-US/vssmartdevicesnative/thread/33bfb706-fd34-4f6a-9679-c6cec4b8fda1/:
"...My solution is not to use DDSCL_FULLSCREEN cooperative level. Use DDSCL_NORMAL..."

Another related link is http://blogs.msdn.com/windowsmobile/archive/2007/08/13/have-you-migrated-to-directdraw-yet.aspx. Sludge said here:
"...When opening a full screen application, there is no way to receive focus change events. For example on the HTC Touch, screen elements aren't drawn correctly when a phone call comes in, or the volume buttons are used..."

It looks that couple of people has problems with not receiving the WM_ACTIVATE message. In order to receive WM_ACTIVATE message the DirectX cooperative level has to be set to DDSCL_NORMAL, not DDSCL_FULLSCREEN.

This looks fine, but it introduces another problem as written here http://msdn.microsoft.com/en-us/library/ms893935.aspx:
"…Processes in NORMAL mode can access primary surfaces but are restricted with regard to flipping, creating complex primaries, and setting display modes…"

On older devices like HTC Wizard(Qtek 9100) where the HW flipping is not supported it is fine but on e.g. HTC Kaiser when using the NORMAL mode, the application crashes when the primary surface is initialized with the error DDERR_NOCOOPERATIVELEVELSET. This is because the HTC Kaiser supports HW flipping, which is detected, and I try to use it, but this is unfortunately not possible in the DDSCL_NORMAL mode. So I have to choose - either try to be phone friendly and handle the WM_ACTIVATE or be game friendly and use the HW flipping…

The application can loose focus from many reasons, e.g. receiving the call, message, Bluetooth transfer, alarm event, calendar notification, going to sleep mode. Even if I receive the WM_ACTIVATE message I need to follow the complex handling:

I call ShowWindow() with SW_HIDE
The game loop is interrupted and application is chilled with the GetMessage()
After the situation is handled (e.g. call) and the user starts the application again, new process is created.
I use the FindWindow() to detect such situation and when I found existing window I set it to foreground by calling SetForegroundWindow() and I exit the new instance

Is there some easier way how to handle it? I'm really pretty new, do I something badly?

Anyway if some Win CE platform developers want to see how to handle such situations in an elegant way - check the DSA (Direct Screen Access) approach used on Symbian platform http://developer.symbian.com/main/downloads/papers/dsa/direct_screen_access.pdf

Every comment welcomed!!

Keypad messages handling on HTC

2009-02-20T06:43:00.000-08:00

I'm facing problems with keypad handling, which I observed on HTC devices (I do not have similar problems with HP iPAQ). The result was either the significant program slow down or in the worse the FPS falls to 0 (so the scene was not updated), depending on how fast I was pressing buttons. I have found that problem happens when user presses buttons quickly in sequence. Then the message processing loop becomes too busy and is never leaved, so the screen redrawing is stopped. Below is the simplified game loop. It consists of two parts, at first messages from the queue are picked and handled and then the scene is rendered. Because the PeekMessage() while loop is too busy with key messages handling, it is not leaved and consequently the Update() function is never called.

while ( isRunning )
{  
  ...
  // 1a. Pick up message
  while (PeekMessage(&msg, NULL, 0, 0, PM_REMOVE))
  {
    //1b. Handle message
  }

  // 2. Renders scene
  Update();
  ...
}

I do not understand, what is wrong with HTC devices, that they handle key press related message so slowly, that it blocks the game loop. I thought this is a common issue, so I tried to search Internet, but the only reference I found was there http://blogs.msdn.com/windowsmobile/archive/2007/08/13/have-you-migrated-to-directdraw-yet.aspx. Guy named Fred Di Sano wrote:
"…Getting input via the message pump is kind of clunky and in some cases (like on various HTC devices) it slows down the app!", but I do not find any other reference..."

I did not found any solution except defining the maximal number of messages that can be processed in the loop iteration:

// Max. messages that are processed in one step
const int KMaxMessagesProcessedInOneStep = 3;

while ( isRunning )
{
 ...
 // Messages processed
 int messagesProcessed = 0;
 // 1a. Pick up message
 while (PeekMessage(&msg, NULL, 0, 0, PM_REMOVE)) 
 {
   //1b. Handle message

   // Check how many messages were processed
   messagesProcessed++;
   if ( messagesProcessed > KMaxMessagesProcessedInOneStep )
     break;
 }

 // 2. Renders scene
 Update();
 ...
}

I believe this is not the best solution, so if anyone can suggest something better I will appreciate it.

Some comments about DirectX on Windows Mobile

2009-02-19T09:27:00.000-08:00

Some comments about DirectX on Windows Mobile

I'm a Symbian developer, who is currently porting a game to Windows Mobile 5.0. I’m trying to use the DirectX and I naively thought it will be easy, but soon I found it is quite fuzzy and tricky. This forced me to make up this blog to share what I have found and get some comments and tips from experienced developers.

1. Hardware flipping support

First surprise was for me that how many devices do not support the surface hardware flipping. This results in fact that I was forced to blit the content of the back buffer instead of just flipping it. I do not know why I expect the hardware support here – probably because I always connected the DirectX with desktop PCs…

I found here http://msdn.microsoft.com/en-us/library/aa454897.aspx:
"…From the above code example, note the check for whether the device is capable of "Flipping" surfaces rather than BLT'ing them to display memory. For the vast majority of emerging Windows Mobile platforms, hardware flipping is not available…"

There is nothing to do with this, just accept it...

2. VRAM

Secondly I thought that the back buffer could be created in the VRAM, but the API call always fails when specifying the DDSCAPS_VIDEOMEMORY flag. When I checked the video memory size it returns me some small values (HTC Wizard 200 (Qtek 9100) – 0 bytes, HTC Kaiser 120 (MDA Vario-III) – 153600 bytes, hp iPAQ hw6915 – 28800 bytes). I was not able to create back buffer in VRAM, so I created it in the system memory, which slows the blitting operation again. I do not have any idea what the VRAM is supposed to be used for or how to use it…

I found here http://www.pocketmatrix.com/forums/viewtopic.php?t=28664:
“…1) From what I've come to understand is that most devices have JUST enough video memory to house the primary surface. Sometimes the device has a bit of extra video ram left which is why the free video memory amount is so small.
2) Creating surfaces in video memory generally doesn't seem to work as there isn't ever enough room there to create a big surface. Maybe small ones that are <= to the free video memory size…”

Any ideas how (if) it can be used?

3. Transparency

I have, without any optimization about 25-30 FPS when the whole scene was rendered (no user interaction). When rendering just one bitmap (size 240x180px) the FPS was about 100. Unfortunately when I started to use the color key to specify the transparent parts of bitmaps the FPS falls to 5.

As written in the discussion here
http://www.codeproject.com/KB/mobile/Caleidoscope.aspx?display=PrintAll&fid=348045&df=90&mpp=25&noise=3&sort=Position&view=Quick&fr=51:
“…I'm doing some blitting with alpha-test (I need to draw some sprites) but when I use the normal blitting with the DDBLT_KEYSRC flag, I lose much in performance (e.g. when I blit a 240X320 bitmap I have 220 fps but when I blit it with DDBLT_KEYSRC flag to lose the background, I get a fps of about 17 ?!)... for the moment, I’m doing the alpha-test myself by locking the surface and checking each pixel, which brings me a fps of about 25 for the same case... I've also tried doing it using GAPI, but I couldn't get a fps higher than 20…”

The only possible solution I can see there is to try to decrease the number of rendered sprites and simplify the way they are displayed plus I could possibly try to make my own routine for transparency handling, but I do not believe this will give me some more FPS. I welcome any hints on that…