After writing up my last blog article about Windows Mobile troubleshooting and logging utilities (see it again here), I was more closely at the lookout for other tools and tricks that might help assist in a similar fashion.. Of course I found some more good additional information and have included it in this round..   Especially the memory management information I don’t think has been that well communicated in the past..

.NET Compact Framework Logging

On Steve Hegenderfer and Reed Robinson’s excellent blog Reed posted a great article about how to enable .NET CF loader logs and what to look out for. Specifically referencing this MSDN information on how to enable the logging: http://msdn.microsoft.com/en-us/library/ms229650.aspx. It is all controlled in specific registry keys on the device to enable 6 different flavors of .NET CF logging: "Interop", "Error", "Loader", "Network", "Finalizer", or "Trace".

The Power Toys for .NET Compact Framework v3.5 download gives you additional tools to make this easier. One is the Remote Logging Configuration Tool:

So the most interesting for non-developers trying to troubleshoot .NET CF applications is probably the "Loader" logging. This is where you can see if the application even makes it off the ground and why. As Reed suggests in the article I mentioned it could be referencing a .NET assembly not present on the device for whatever reason..

Additional details on how to read the "Loader" logging can be found here: http://msdn.microsoft.com/en-us/library/ms229667.aspx.

File System Logging

This is a type of extreme logging that can really slow down a working operating system. But it can also show you exactly what is going on at the file I/O level. Specifically what files are being accessed or written to. This could be useful to trace back missing files or folders, or figuring out the last file access a specific application did before failing.

I only recently found a tool called MobileMon v0.5 by Brian Dunn. His website, http://www.mobilmon.com/, has more information and you can download the .CAB file there.

Basically you can install and run it in the background while it logs file activity.

Once you are done you can save it to a log file. Be aware however that the file name "mobilmon.log" may be hard to open on the device itself unless you install a tool (Like Voyager or Total Commander) to rename the file to mobilmon.txt. Then you can open it with the native Word Mobile.

Memory Management and Monitoring

Another important area of concern for current Windows Mobile troubleshooting is available memory on the device.  Memory leaks, multiple running applications, and garbage heaps can all attribute to doing frequent soft-reboots to get a device functional again. A little known fact that I wasn’t fully aware of is that only 32 applications (actually processes) can run at the same time and each can at a maximum access 32mb of virtual memory..

An excellent resource of a virtual memory management overview is William Blanke’s article: http://www.codeproject.com/KB/mobile/VirtualMemory.aspx

In it he also has a small (12Kb) Virtual Memory tool (must register to download, the compiled .exe in included with the source code) you can run and visually see available memory (in red) for each of the 32 process slots.

Issue #1: One key thing apart from seeing how many of the slots are being used and if they are full, is finding the “device.exe” process. This process is responsible for loading up all the device drivers and William points out the potential issues if memory is low for this slot. Specific device features may simply not work.

Issue #2: Another area of concern could be applications that load up .DLL files. These can be loaded up in *any* processing slot and can be accessed by any process. This can be bad if your process or application running in the slot needs the memory and doesn’t use the particular DLL.

However William doesn’t address that in Windows Mobile 6.1 specific changes were made to better accommodate DLL files over 64Kb. These will now be loaded into specific slots higher and away from the process slots. Thus freeing up application space and reducing this potential worry. Please see more information on this 6.1 feature from Doug Boling here.

How sure if anything has/will change in Windows Mobile 6.5 as of yet. What we can look forward to is Windows Mobile 7.0 (which is based upon Windows CE 6.0) and it’s larger scale advanced memory management, explained in more detail here or here. But basically a little like Windows XP, and a limit of 32K processes and 2GB per process, compared to 32 and 32Mb per process.

Issue #3: Careful on the usage of storage cards to install or run applications from. If the device goes into hibernation or sleep mode, it could power down the storage card and render any application housed there non-functional. See more tips here.

Some older reference links on Windows Mobile memory management: 
- RAM, ROM and Task Managers 
- How WM 5.0 Shell Handles Low Memory Situations 
- Memory Management on WM 6.x 
- MSDN Webcast: Memory Management for Windows Mobile
- DumpMem Utility

If you are using a Motorola/Symbol ruggedized device you also may want to ask your Motorola rep about their “Private SDK” and a tool called the “Remote Memory Viewer”. It may also be beneficial as Raffaele Limosani states here..

Hope this article further assists in troubleshooting Windows Mobile issues you might run into!

|\\arco..
http://marco.blogsite.org

A quick heads up on some interesting new Microsoft webcasts coming up early next month on Windows Mobile Device Management and Security that may be of interest to many of you:

TechNet Webcast: Windows Mobile 6.1 and Mobile Device Manager 2008: The Gateway to Your Corporate Network (Level 200)
Tuesday, April 7, 2009
10:00 A.M.-11:00 A.M. Pacific Time

TechNet Webcast: Management Lockdown of Windows Mobile Devices (Level 300)Thursday, April 9, 2009
11:30 A.M.-12:30 P.M. Pacific Time

Register now and get it on your calendar!

|\\arco..

I’ve been compiling a running alphabetical list of which devices now have official supported upgrades available for them since the summer of 2008. This may be useful for many of you as well implementing SCMDM and researching which devices are compatible. 

Several links fixed, and several devices purchased with WM 6.1.x builds now listed as reference as well. Interesting to see the slow uptake of devices having the 6.1.4 build finally that has the Internet Explorer Mobile 6 (IE6on6).

Included is the specific OS/AKU build for each device for SCMDM 2008 SP1 support.

If you know of others, updates or corrections, please let me know! 

|\\arco..
mnielsen(at)enterprisemobile.com
http://marco.blogsite.org

Windows Mobile security best practices are a key component of Enterprise Mobile’s expertise and services, but recently we’ve been much more vocal about it! First off, there’s the excellent WM Application Security White Paper that my colleague Dave Field just published. Here’s a brief synopsis:

This technical paper recommends how enterprises can take advantage of the powerful security features of Windows Mobile to defend against malicious and unsupported application use. Taking a very pragmatic approach, the paper describes how various features work and how to implement them to protect devices based on Windows Mobile 5.0, 6.0 and 6.1.

Go ahead, download the WM Application Security White Paper now, I highly recommend it for any IT professional who’s interested in Windows Mobile security. Dave has put incredible detail into this paper, making it invaluable for an organization who is currently using (or planning to deploy) Windows Mobile devices and applications.

Next up, there’s an interesting Network World article by John Cox about Mobile browser security that Dave and I are quoted in.  The article focuses on the impact that a new generation of mobile web browsers will have on how enterprise IT organizations handle mobile device security.  John sums up the three key areas that enterprises should focus on:

IT departments, according to experts, need to focus on three areas: assessing the security architecture and features in the mobile browser and the underlying operating system; working with users on smart and safe browsing practices; and creating a solid handheld device management system.

In fact, choosing a mobile platform with a strong and flexible security model in hand with a solid device management system can help you minimize the headaches that users have to endure. With those first two handled, educating users on smart and safe browsing practices is something that is applicable to both traditional desktop web browsers as well as the new crop of full-featured mobile browsers. Read the full article, titled “Mobile browsers bring new security headaches” now for more information.

As part of supporting Windows Mobile in an enterprise environment, one of the things that often will come up is what tools are available for troubleshooting..

One powerful tool that has been around since the dawn of the first computer programs is logging. Here are a few important Windows Mobile logging tips that can be extremely helpful and save your day:

Exchange ActiveSync Device Logging

Nice write-up from Vik Thairani on how to enable the verbose logging on Windows Mobile for Exchange ActiveSync troubleshooting:
http://blogs.technet.com/vik/archive/2008/12/04/setting-up-verbose-logging-in-windows-mobile-and-parsing-logs.aspx

The log is saved in text file in the \Windows\ActiveSync folder starting with “serverlog” and a sequential number.

SCMDM Device Management Logging

With MDM Connect Now Tool, you can enable or disable various types of logging as necessary. To enable enrollment logging on a device using MDM Connect Now Tool, select Menu, and then select Logging.

For information about MDM Connect Now Tool, see the MDM Resource Kit Tools at this Microsoft Web site: http://go.microsoft.com/fwlink/?LinkId=127030.

1. EnableNodeMon log – If this option is checked, the system generates a log file at \NodeCache.txt.
2. Enable OMADM log – If this option is checked, the system generates a log file at \deviceupdate.log.
   See http://technet.microsoft.com/en-us/library/dd252860.aspx for some information on what this log can show.
3. Enable Enroll log – If this option is checked, the system generates a log file at \deviceupdate.log.
4. Enable Scheduler log – If this option is checked, the system generates a log file at \Application Data\Logs\Scheduler.txt.
5. Enable alerter log – Generates a log file at \deviceupdate.log.
   If this option is checked, the system enables the following values:
   • Alerter – Search for “Rejecting packet” or “Successful push packets” in the log.
   • Nodemon InitSession
   • Nodemon configuration service provider
   • Software Distribution
   • TDET settings

Please see http://technet.microsoft.com/en-us/library/dd261878.aspx for additional details on these logs.

SCMDM VPN Device Logging

The MDM VPN Diagnostics Tool can be downloaded from http://go.microsoft.com/fwlink/?LinkID=127030.

To enable and disable Mobile VPN logging on your Windows Mobile device, run the MDM VPN Diagnostics Tool and follow these steps:

1. On the Start page, select Menu.
2. Select Logging.
3. Select Enable or Disable.

MDM VPN Diagnostics Tool includes a Log Browser for viewing the VPN Service log file located at \Application Data\Logs\ipsecvpnpm.txt.

Network Traffic Device Logging

Sometimes the best recourse for technical troubleshooting is determining what is going on on the network level. On a Windows Mobile device this can also be accomplished.

The Microsoft Windows Mobile Network Analyzer PowerToy v1.0 can be directly downloaded from: http://www.microsoft.com/downloads/details.aspx?familyid=081c6401-49d4-4506-a03b-c41bc76c2f51&displaylang=en.

If you have a storage card inserted, Network Analyzer will write all logs under \Storage Card\NetworkLogs. If there is no \Storage Card, it will write all logs under \NetworkLogs.

To capture the network traffic (NetMon) log for analysis, run the start analyzer script in the Program directory. Run the stop analyzer script to stop the network logging.

Then you can view the .cap file in your network protocol analyzer of your choice to properly decipher all the information. I highly recommend the freebie WireShark efforts from http://www.wireshark.org/.

An example (from http://technet.microsoft.com/en-us/library/dd252860.aspx) to troubleshoot SCMDM VPN issues on a Windows Mobile device:

1. Install the Windows Mobile Network Analyzer PowerToy.
2. Install MDM VPN Diagnostics Tool.
3. Start MDM VPN Diagnostics Tool, select Menu, and then disable VPN.
4. Make sure that you can browse the Internet using Internet Explorer Mobile through your WiFi or Mobile Operator (carrier) data connection.
5. Start the Windows Mobile Network Analyzer PowerToy to capture network traffic on the device.
6. Enable VPN using MDM VPN Diagnostics Tool.
7. When the VPN connection fails, stop capturing network traffic, and save the trace file.
8. View the VPNDiag report and the ipsecvpnpm.txt file from the device.

For more information, view the readme file that accompanies the Windows Mobile Network Analyzer PowerToy.

|\\arco..
mnielsen (at) enterprisemobile.com
http://marco.blogsite.org

This is a brand new feature of SP1 of great interest in an enterprise implementation. This mimics the similar Exchange and Windows Mobile device functionality, but without the need for any Exchange requirements. With this feature end users who have forgotten their device password or PIN, can recover (without wiping the device) and set a new device password or PIN. In this posting I will dive a little deeper and show how this all works on both the server and client side.

Overview

As nicely stated in the MDM Password Reset Client v1.0 download overview:

“MDM Password Reset Client provides a .cab file that you install on Windows Mobile 6.1 devices enrolled in MDM so that users can use the password reset feature in MDM. Password reset in MDM 2008 Service Pack 1 (SP1) enables a user who has forgotten his or her Windows Mobile device password to reset it by using MDM.

Password reset is supported on Windows Mobile 6.1 devices, starting with version 6.1.4. To use the feature, you must install the .cab file on the user’s Windows Mobile device as well as enable the feature in MDM by using Group Policy.

To reset the device password, the user chooses the password reset option, resets the device password, and then enters a one-time recovery password on the device to complete the process. The recovery password is stored on MDM servers and retrieved by the user when she or he has forgotten the device password.”

What is required?

Even though the client patch description mentioned above states it is first supported on Windows Mobile 6.1.4 or above device, the patch appears to install on some of my 6.1.1 devices. But “your mileage may vary” (YMMY) as they say..  The patch, available here, can be manually installed, but with MDM handy why not deploy it it out directly!  Please note the installation failures on the devices that are below the 6.1.1 levels.

You also need the SCMDM 2008 SP1 installation on the back-end. Especially the changes on the DM server, SQL tables, and Self Service Portal (SSP) if you wish to use that for retrieving the reset password.

How it works:

After the client patch on the devices is installed and the device locked with a PIN, triggers a local generation of a password reset key. After 2 cycles of traffic to and from the Device Management server, that recovery password will have uploaded to the SCMDM side and be available for use.  This can be verified with a cmdlet or on the MDM console by seeing that the “Display Recovery Password” action is no longer grayed out on the right hand side of the screen when a managed device is selected:
 

More details can also be found here on the overall user experience of this feature: http://technet.microsoft.com/en-us/library/dd252841.aspx

Client Functionality

These are actual screen-shots of a managed device that has the client patched installed.

In a locked state, the “Reset Password” option is no longer grayed out. Suggesting that the password reset key has been uploaded and ready to use:

After the “Reset Password” option is selected, a confirmation that the user can indeed retrieve the recovery password from an administrator or help desk.

It will then let the user create a new password. Using the same requirements that might have been enforced to the device.

Now the user must contact the administrator or help desk. In this example the administrator clicks on the “Display Recovery Password” in the MDM console and is shown the 20 digit Recovery Password that the device has uploaded into the MDM database.

The user must type in the 20 digit recovery password to validate the new password.

If there is a match with the recovery password stored on the device, the new password is granted and the device is unlocked!

Instead of the MDM console, the MDM Self Service Portal (SSP) could have been used. It also has a “Display Recovery Password” button at the bottom which will display the 20 digit recovery password:

The Password Recovery feature in the SSP is selectable by the administrator to be made available on the web site just as the Device Wipe and Device Enrollment features. Please see more information available here: http://technet.microsoft.com/en-us/library/dd261796.aspx.

Password Recovery References

SCMDM Cmdlets: http://technet.microsoft.com/en-us/library/dd261726.aspx
SCMDM User Experience: http://technet.microsoft.com/en-us/library/dd252841.aspx
Windows Mobile 6.x AKUs: http://myitforum.com/cs2/blogs/mnielsen/archive/2009/01/31/windows-mobile-6-x-akus.aspx
Windows Mobile 6.1.x Upgrades and Build Levels: http://myitforum.com/cs2/blogs/mnielsen/archive/2009/01/24/windows-mobile-6-1-x-upgrades-now-available.aspx

|\\arco..
mnielsen (at) enterprisemobile.com

A lot of discussions within IT organizations are about security, and how the approved security policies must be executed and implemented. Traditionally it is not the same group of the staff that has mandated the security policies that has to implement tools and processes to have them executed. But I have seen how this “disjointed camp” trend is slowly becoming better in many organizations.

The focus of this posting is to highlight some of the options available that I have run across recently on the question of Windows Mobile security and encryption. In particular what Windows Mobile 6.1 brings and potential issues you might encounter depending on your security policies and requirements.

Native Device and File Encryption

Starting in the Windows Mobile 6 release there was native support for device and file encryption. In Windows Mobile 6.1 this was further enhanced with additional features to handle storage cards inserted into the device. This could be triggered from Exchange 2007, from System Center Mobile Device Manager (SCMDM) in a Group Policy Object (GPO), or even from a 3rd party tool on the device like Andreas Helland wrote (see http://mobilitydojo.net/2008/11/19/update-dojocrypt-goes-10/). Basically specific registry keys needs to be flipped to activate the appropriate features.

The encryption code is built into the Windows Mobile operating system so there is low over head.  The encryption on the storage card is upon write, so existing data on the card is not encrypted unless re-written. In WM 6.1 you can also specify exclusions/inclusions of directories . This can be handy to only encrypt e-mail or critical folders with line-of-business applications.

Data Recovery

Some companies require that the encrypted data can be retrieved. This may go against the reasoning behind encryption you say, but depending on your security mindset and corporate data ownership a necessary evil.  Say some important employees have important data on an encrypted storage card or device and you need to access the data, either for support or legal reasons, with or without their co-operation.

On desktops and servers there are some processes to accomplish this with encryption Key Escrow or Recovery using storage of the encryption key elsewhere. This of course brings other security risks into effect. I know the Windows Vista/Windows 2008 BitLocker technology for example accomplishes this through the Active Directory.

But on the native Windows Mobile 6 and 6.1 operating system this was not prioritized as a necessary component of the latest OS release as it would have probably required more work and back-end integration. But who knows what the future could bring if enough enterprise customers ask for it! (hint hint, this is where you have a say back to Microsoft!)

Thus if you wipe a Windows Mobile device, either from Exchange 2007 or System Center Mobile Device Manager (MDM) or other means and the encrypted storage card that was previously encrypted within the device was taken out beforehand, there is no way to read the encrypted data on the card again. It can only be read on the device it was encrypted with.

Jason Langridge’s blog entry here lays it out nicely: http://blogs.msdn.com/jasonlan/archive/2007/03/16/storage-card-wipe-and-encryption-what-s-the-deal.aspx 

- and a reference from the Windows Mobile product team themselves from their encryption FAQ:
http://blogs.msdn.com/windowsmobile/archive/2007/03/26/windows-mobile-6-storage-card-encryption-faq.aspx

Key Recovery

If your security policies require encryption key recovery processes, you may need to look at 3rd party solutions. These will of course bring an additional cost, but may also add additional security features.

Some or most solutions work around the issue by creating their own encrypted file “volumes” and backup the known key used. Thus not using the default file encryption implementation.

Possible products and links: 

Aiko Solutions SecuBox: http://www.aikosolutions.com/products/secubox-for-pocket-pc/articles/secubox-encryption-vs-windows-mobile-6-encryption/
GuardianEdge Smartphone Protection: http://www.guardianedge.com/products/smartphone-security.php
CheckPoint Pointsec Mobile: http://www.checkpoint.com/products/datasecurity/mobile/index.html 
McAfee SafeBoot: http://www.mcfee.com 
Mobile Armor DataArmor: http://www.mobilearmor.com/dataarmor.php 
Credent Mobile Guardian: http://www.credant.com/products/cmg-enterprise-edition.html 
WinMagic SecureDoc Mobile Edition: http://www.winmagic.com/solutions/securedoc-pda.html
PGP Mobile: http://www.pgp.com/products/mobile/index.html

No default workaround

Some good technical explanation and background from my colleague and resident expert Mr Dave Field (CISSP) from Enterprise Mobile on the technical aspects of using the native Windows Mobile 6.x encryption and why a workaround isn’t currently possible with the default implementation of encryption:

“The problem is that both storage card encryption and device main memory encryption is performed using keys that are auto-generated and auto-encrypted by DPAPI.  The DPAPI system key and user key are encrypted using device-specific entropy.  The user key on WM 6.1 used for encryption adds the device lock PIN/PW as entropy.  So, even if you went and found the keys in memory and uploaded them to the infrastructure, you wouldn’t be able to decrypt them using some shared password.  There is no function available to decrypt the key and provide the output.  DPAPI only decrypts the  key into memory as part of encryption/decryption operations.  DPAPI has no archiving function and is not tied into Active Directory.  When using EFS or even enrolling a certificate, the keys can be archived using active directory. 

Even if we found the registry keys for the auto-generated DPAPI keys and stored them centrally. We couldn’t re-use them elsewhere if we replaced them and used the same user PIN/PW on the new device. This is because there are a number of device characteristics used for entropy as well as the user PIN/Password.  The entropy points are not advertised for the obvious reasons…”

Wrap-up

Please comment if you have different experiences, feedback or interesting views on these issues!

References of possible further interest on this topic:

Why Device Lock PIN/Password must be configured with Windows Mobile 6.1 Device Encryption:
http://blog.enterprisemobile.com/2008/06/why-device-lock-pinpassword-must-be-configured-with-windows-mobile-61-device-encryption/
Windows Data Protection API (DPAPI): http://msdn.microsoft.com/en-us/library/ms995355.aspx
Older Mobile Encryption paper: http://www.sans.edu/resources/student_projects/200612_001.pdf
Keep Mobile Devices Safe With Encryption (Nov 2007):
http://www.informationweek.com/news/mobility/security/showArticle.jhtml?articleID=202803981&pgno=2&queryText=&isPrev=

|\\arco..
mnielsen (at) enterprisemobile.com

If you haven’t already I highly recommend that you upgrade your Live Search to the latest version previewed at CES last week: http://news.cnet.com/8301-1035_3-10141820-94.html

The biggie being the ability to locate your approximate location without a GPS built into your device.  Also predictive text/word completion has been added with a weight on previous hits you have done. Upgrade from within the app as shown below, or directly download it from http://wls.live.com on your device.

For enterprise deployments of the Microsoft signed Live Search .CAB file through SCMDM 2008 please see this article: http://blog.enterprisemobile.com/2008/04/software-distribution-with-mdm

|\\arco..

I’ve been compiling a running alphabetical list of which devices now have official supported upgrades available for them since the summer of 2008. This may be useful for many of you as well. New entries are in red..  Sorted by mobile operator/OEM and now made it more condensed as well!

Now included the specific OS/AKU build for each upgrade for SCMDM SP1 support.

If you know of others, updates or corrections, please let me know!

Update Dec 15, 2008: * Thanks to Wayne Philips of Airloom for these build numbers!

|\\arco..
mnielsen(at)enterprisemobile.com
http://marco.blogsite.org

Hello from sunny LA.  Yes, all the natives thought they had seen a pre-halloween ghost when this seattle native hit the streets for the Microsoft Professional Developers Conference 2008 (PDC2008).  As usual, Microsoft has some new, cool products and technologies in the developer space that hit in a big way.  You’ve probably been seeing headlines covering Windows 7 and Microsoft “Azure”. 

Here is my very short, sweet summary of this new stuff.  First, Windows 7:

• Winows 7 will fix all that stuff that caused negative reviews on Vista. 
• It also has a lot of great, new shell features that will positively impact the experience for the everyday user.  Just the taskbar improvements alone are pretty cool.
• UAC will still be there, but in a more flexible format for configuration and they have decided to except more operations from prompting.  However, Microsoft continues to be hardcore about forcing all apps to the do the right thing and operate under “standard user”.  The stats do show that most developers are getting the message and a lot of progress has been made.  As a user, UAC drives me up the wall.  As a security person, it’s the right thing to do.

The more exciting (to me) and new technology has to do with the new Microsoft services strategy which includes “Azure” and the “Geneva” server.  This technology will catalyze  two, important business scenarios that really need to get over the hump:  (1) B2B connectivity in which there are many enmeshed partners sharing a workflow and (2) hosted services for enterprises (not just small orgs).

Microsoft has a big cloud in the sky and plays traffic cop for all services that register to the Microsoft “Services bus”.  But, the bus supports some serious authentication and authorizatio through use of WS federation and SAML tokens.  And, part of the offering is SQL services which equates to a SQL DB that is up in the cloud and protected by the aforementioned authentication and authorization.  So, you can support some great B2B scenarios:

• Partners that all need access to a workflow, but need a slightly different type of data for the same workflow transactions
• Partners that all use a different directory or authentication type can still positively identify into one cloud
• Eventing enables store and forward of transactions when one particular partner connectes to the service.

If you are a small company and you are interested in advertising your service, click into the Microsoft service bus and you just go a free advertisement to services consumers.

But…that’s not all.  The biggie is Geneva because it creates a super easy to setup and configure Enterprise Service connector for Active Directory.  This could enable hosting of an internet-based service to a company with an internal Active Directory.  There is a question here of whether the company will accept the Microsoft EULA for connecting to the Services bus and whether their security policy will accept their authentication getting proxied through the Microsoft Federation Gateway in the cloud.  But, the good news is that all the authentication against the hosted service is handled by the Microsoft Service Connector which is located on the company premises.  It reminds me a little bit of ADSI, but better.  If the company doesn’t want to accept the Microsoft EULA, they can set up a B2B direct to their partner ( the hosting provider) who will have the Federation Gateway  (Geneva) handling “claims”.

We still have 1.5 more days here, but I think all the big news has already popped.

Dave Field, CISSP, MCP