Wednesday, August 30, 2006

2D barcodes in a sequence

There are a bunch of really cool applications out there based on the 2 dimensional barcode scanning with camera phones. More so in Japan (QR code) as the reader comes pre-installed on the devices.

I have been thinking about this for quiet some time now and so far I havent found anyone doing this. How about using barcodes for downloading content directly on the phone? No, not by opening the WAP browser and then using the mime type of the content to govern the actual download and installation. How about breaking up the content file and encoding it into a bunch of 2-d codes and then having the user download the content by scanning all the codes in succession. Could be done? YES!!! (technically speaking).

Example:
A user wants to download a string "1234" on his phone :-). So we create 4 codes with the values '1','2','3' and '4'.
1
234
These individual codes could then be combined into one single fixed delay sequence that can be captured by camera phones. Most of the barcode reading applications already support the Append mode and it shouldnt be a strecth to incorporate a streaming (using this word for the lack of other words) protocol in the barcode reading process.
Theoretically, the application could then assemble all the data by reading these individual QR codes to create the complete content file. Once the file has been created, the application will read the mime type and launch the registered content handler. Hmmm... sounds too easy. Well, there will be a bunch of issues that will have to be solved, specially those related to the frame delay in the sequence and the type of content that can be encoded. The benefit, download content without Data service :-).

Monday, August 28, 2006

BREW: ICamera Interface

I have been meaning to write this post for quiet some time now and finally got around to finishing it. I am including a few tid bits in this post about the BREW ICamera Interface. The post is not meant to be a tutorial but could provide you a starting block if you are looking to add the camera functionality into your BREW Applet and then some.
For tutorials, I recommend the following:

  • Devx:Camera-enable Your Applications with BREW's ICamera APIs by Ray Rischpater.
    A nice short tutorial to get you started with the ICamera Interface. In case the above link is broken, try searching on Google or Devx.com for the article.
  • BREW forums
    If the above link does not work, search for ICamera and you will get plenty of posts with issues about the interface and also notes on how to solve them.
Ok ... Now that you have the links, lets check out the ICamera interface.

Determine Device Support:
The ICamera interface was introduced in BREW version 2.1. The support for the interface was slow initially but now pretty much any camera phone coming out supports the interface. To check for support before actually buying the device, take a look at the BREW specs (Data Sheet) for the device available through the Qualcomm BREW developer website. Usually there is a section called "Camera" in the spec sheet dedicated to camera access that provides information about:



  • Camera Available to BREW Applet.
  • Resolution (usually the native camera resolution).
  • Access to native photo directory.
  • Path of the native photo directory.

Creating the ICamera Interface Instance:
Let's start with the header files:

  • AEECamera.h: Provides the definitions of the ICamera interface and will have to be included in your build.
  • AEEBitmap.h: Provides the definitions of the IBitmap interface. The frame captured by the camera is returned as IBitmap, so this is important.
  • AEEMimeTypes.h: Contains the definitions of the mime types etc and you might need this if you are planning on taking snapshots etc...
To create an Instance of ICamera, we resort to the trusty ISHELL interface.
ICamera *m_pICamera = NULL;
int nRet = EBADPARM;
nRet = ISHELL_CreateInstance(pMe->iShell, AEECLSID_CAMERA, (void **)&m_pICamera);
If access to camera is supported and an instance was instantiated, the function returns SUCCESS. Other common codes returned include:
  • EBADPARM: Check the pointers you are passing to the function.
  • EUNSUPPORTED: You should really check the specs on the device before trying to instantiate the ICamera interface.
  • EPRIVLEVEL: This is weird but happens on certain devices. To avoid getting this return code, you can try adding a dependency for the camera in your MIF. More details on this here.
Once we have our instance of ICamera, the next thing to do would be to register a Callback function. All a developer has to do is to have a function that corresponds to the PFNCAMERANOTIFY specification.

typedef void (*PFNCAMERANOTIFY)(void* pUser, AEECameraNotify * pNotify);

The implementation of ICamera interface is Asynchronous similar to the other interfaces like IMedia and IWeb. The operation of the camera and the access to it revolves around the BREW layer notifying the Applet via the callback function. More information on this can be found in the BREW SDK documentation.The RegisterNotify function is used to register the Callback function.
/*Registering the callback function.
m_pICamera: Instance of ICamera created above
_CameraNotify: Callback Function
pMe: Pointer to the instance of the Applet.
You can pass anything here. This ptr will be returned in the
callback as the void pointer pUser. */
nRet = ICAMERA_RegisterNotify(m_pCamera, _CameraNotify, pMe);
For more details on the AEECameraNotify structure, refer to the BREW SDK documentation. I usually use the following members of the structure to determine the corresponding action in the callback function.
nStatus : The current status.
nCmd and nSubcmd: Gives an indication of the current camera mode.
//Generic Implementation of the callback function.void _CameraNotify(void * pUser, AEECameraNotify * pn)
{
//Get a handle to the reference pointer.
MyObj* pMe = (MyObj *) pUser;
if(!pMe !pn)
return;
//Handling based on Status passed.
switch(pn->nStatus)
{
case CAM_STATUS_START:
/*Sent when Preview or Record operation is started.*/
if(pn->nSubCmd == CAM_MODE_PREVIEW && pn->nCmd == CAM_CMD_START)
{
/*preview Mode has started.*/
}
if(pn->nSubCmd == CAM_MODE_SNAPSHOT &&pn->nCmd == CAM_CMD_START)
{
/* This is the first callback recieved when you initiate the Snapshot mode.*/
}
break;
case CAM_STATUS_DONE:
/* [Preview/Record/SetParm/GetParm/EncodeSnapshot]
Operation completed successfully.
For RecordSnapShot, pData = TRUE/FALSE = > Defered encode enabled/disabled
*/
if(pn->nSubCmd == CAM_MODE_PREVIEW && pn->nCmd == CAM_CMD_START)
{
/*Preview Mode has completed.*/
}
else if(pn->nSubCmd == CAM_MODE_SNAPSHOT && pn->nCmd == CAM_CMD_START)
{
/* Snapshot has been taken. In defer mode, the bitmap captured can be accessed here.
In default mode, you can wait for the snapshot encoding to finish.
if(pn->nSubcmd = = CAM_CMD_ENCODESNAPSHOT)*/
}
if(pn->nCmd == CAM_CMD_SETPARM)
{
/* When a parm such as Zoom etc has been set.*/
}
break;
case CAM_STATUS_FAIL:
/* [Preview/Record/SetParm/GetParm/EncodeSnapshot]
Operation failed pData = CAM_EXXX error code.
You can use the pn->nCmd andpn->nSubCmd to see exactly what operation failed. */
break;
case CAM_STATUS_ABORT:
/*The last operation was aborted. The camera is now in the ready state.*/
break;
case CAM_STATUS_FRAME:
/* [Any] Frame captured by camera. The frame can be accessed using ICAMERA_GetFrame()*/
break;
case CAM_STATUS_PAUSE:
/* [Preview/Record] Record movie paused.
This status is returned when you call ICAMERA_Pause()*/
break;
case CAM_STATUS_RESUME:
/* [Preview/Record] Record movie resumed.
AEECameraNotify::pData is IBitmap pointer representing the snapshot*/
break;
case CAM_STATUS_DATA_IO_DELAY:
/* [Preview/Record] Operation being delayed by data i/o access*/
break;
case CAM_STATUS_SPACE_WARNING:
/* [Record] Memory available to store recording running low.
To Do: Check if this applies to Snapshots too. Havent really checked this yet.*/
break;
}
}
Now that we have the different status codes that are returned in the Callback, lets check out the modes.

Starting the Camera (PREVIEW Mode)
J2ME developers will find the approach a bit different, there is NO Videocontrol :-). The developer is responsible for getting the captured frame from the instance and then blitting it on the LCD. The approach is very similar to the Series 60 API's and maybe microsoft smart phones (I am not sure).
Things to do:

  • Set the Size of the Preview Frame.
  • Start the Preview Mode.

Offcourse, there are bunch of Optional things you can do before starting the preview mode but remember, just cause its stated in the API, doesn't mean its supported on the phone.

The BREW API provides the developer a way to query the supported resolution for rendering the preview frames. Again, not all phones will support this method. The simplest way to set the size for the preview frames is to use the device LCD size. Since the frame has to be rendered on the device screen, might as well use this size(Japanese phones from KDDI do provide a preview frame of size larger than the LCD. But the frame is scaled when you blit it on the entire screen.). Using the device screen size gives the default preview size for the device.

static void initDisplaySizes(cameratest * pMe)
{
AEESize *ltSize;
int loop;
boolean bRet= FALSE;
int nRet = EFAILED;
//Get the Resolution available for the Preview Mode.
ltSize = (AEESize *)CAM_MODE_PREVIEW;
nRet = ICAMERA_GetDisplaySizeList(pMe->m_pICamera, if(SUCCESS != nRet)
{
// There was an error in getting the sizes. use the screen size.
return;
}
// Got the available sizes.
// Run through the loop and get the
// best resolution possible.
for(loop = 0 ; ltSize[loop].cx != NULL && ltSize[loop].cy != NULL ; loop++)
{
// Width: ltSize[loop].cx
// Height: ltSize[loop].cy
}
}
To start capturing and displaying the frames:
static void cameratest_DisplayCameraPreview(cameratest * pMe)
{
int nRet;
AEESize az;
// Setting the Display Size. Using Screen Size.
az.cx = pMe->cxScreen;
az.cy = pMe->cyScreen;
ICAMERA_SetDisplaySize(pMe->m_pICamera, &az);
// Start the Preview mode.
// This call will start the messages to be
// sent to the callback function.
nRet = ICAMERA_Preview(pMe->m_pICamera);
}
Once the Preview mode has started, a notification with the status CAM_STATUS_START is sent to the callback function. In case of error, a corresponding error status is sent. The individual frames captured by the camera are sent with the status CAM_STATUS_FRAME. The developer must implement logic here to display the frames on the screen.
void _CameraNotify(void *pUser, AEECameraNotify *pn)
{
cameratest * pMe = (cameratest *) pUser;
if(!pMe !pn)
return;
switch(pn->nStatus)
{
case CAM_STATUS_START:
//Preview mode has started. Do any applet specific
// stuff here.

break;
case CAM_STATUS_FRAME:
// A frame has been captured by the camera and
// is now available for rendering on the LCD.
IBitmap * pFrame; AEEBitmapInfo bi;
//Get the captured frame
ICAMERA_GetFrame(pMe->m_pICamera, &pFrame);
if (!pFrame)
break; //Error
//Get the bitmap info...
//IBITMAP_GetInfo(pFrame, &bi, sizeof(bi));
//Blitting the frame on theLCD from (0,0)
IDISPLAY_BitBlt(pMe->pIDisplay,0, 0,pMe->cxScreen, pMe->cyScreen, pFrame, 0, 0, AEE_RO_COPY);
// Release the Bitmap instance
IBITMAP_Release(pFrame);
//Update the Display
IDISPLAY_Update(pMe->pIDisplay);
break;
case CAM_STATUS_DONE:
// The camera has stopped the preview mode
// and has entered the ready state.

break;
}
}
To stop the camera in preview mode, use the ICAMERA_Stop() function. This will send the CAM_STATUS_DONE status to the callback function.

Adjusting Parameters:
Now that the camera is running in the preview mode, lets take a look at some of the control parameters available to the developer. Digital Zoom, Brightness, Contrast etc. Are some of the control parameters that are available through the BREW api. For the complete list check the SDK documentation. The SDK provides wrapper functions for adjusting these parameters and also checking if they are supported (ICAMERA_IsBrightness). I personally prefer using the ICAMERA_SetParm and ICAMERA_GetParm methods eventhough I am probably replicating code :-).
#define _CAM_MOV_UP 1 // Move one Step Up
#define _CAM_MOV_DOWN 2 // Move one Step down
#define _CAM_MOV_HIGH 4 // Move to highest
#define _CAM_MOV_LOW 8 // Move to the lowest

#define ADJUST_QUALITY(po, pDir) adjustParm(po, pDir,CAM_PARM_QUALITY)
#define ADJUST_ZOOM(po, pDir) adjustParm(po, pDir,CAM_PARM_ZOOM)
#define ADJUST_BRIGHTNESS(po, pDir) adjustParm(po, pDir,CAM_PARM_BRIGHTNESS)
#define ADJUST_CONTRAST(po, pDir) adjustParm(po, pDir,CAM_PARM_CONTRAST)
#define ADJUST_SHARPNESS(po, pDir) adjustParm(po, pDir,CAM_PARM_SHARPNESS)

static int adjustParm(cameratest * pMe, int pDirection, int parmID)
{
int nRet;
AEEParmInfo pi;
int32 p1;
int adj = 0; //Adjusted Value
//Get the Parm Value
nRet = ICAMERA_GetParm(pMe->m_pCamera,(int16) parmID, &p1,(int32 *)&pi);
/**@TODO: Handle CAM_PENDING */
if(SUCCESS = = nRet)
{
//Calculate the next step
switch(pDirection)
{
case _CAM_MOV_UP:
adj = pi.nCurrent + pi.nStep;
break;
case _CAM_MOV_DOWN:
adj = pi.nCurrent - pi.nStep;
break;
case _CAM_MOV_HIGH:
adj = pi.nMax;
break;
case _CAM_MOV_LOW:
adj = pi.nMin;
break;
default:
return EUNSUPPORTED;
}
//Set the New Value
nRet = ICAMERA_SetParm(pMe->m_pCamera,(int16) parmID,(int32) adj, 0);
}
return nRet;
}
To increase the digital zoom all I have to do is call ADJUST_ZOOM macro. I am uploading a zip file containing the source code of the application for the preview mode and also the ARM binaries. My development environment is Microsoft Windows .NET 2003. In case the link below does not work, shoot me an email and I will send you the zip file.
Download Sample Application 1

Taking Pictures:
The most obvious use of the ICAMERA interface in any applet is to take snapshots. The SNAPSHOT mode of the interface allows you to do this. One thing to remember is that the ICamera instance MUST be in the READY state for the operation to succeed.
Things to do:

  • Make sure the camera is in READY state. If you are in preview mode then call the ICAMERA_Stop() function to get there.
  • Set the Media data (ICAMERA_SetMediaData()).
  • Set the size of the picture to be taken (ICAMERA_SetSize()).
  • Set the encoding for the image (ICAMERA_SetVideoEncode()).
  • Adjust the quality of the picture to be taken (ICAMERA_SetQuality()).
  • Initiate the picture taking operation (ICAMERA_RecordSnapshot()).

Taking a snapshot is a 2 stage operation, the first step is the actual capturing of the frame and the second step deals with encoding the snapshot. Another mode, ENCODESNAPSHOT maintains the camera state when the encoding is being done.


static void takePicture(cameratest * pMe)
{
AEESize sz;
// 1) Set Media Data
pMe->md.clsData = MMD_FILE_NAME;
pMe->md.pData = "snap.jpeg" ; //Store the picture in the applet directory
pMe->md.dwSize = 0 ;
ICAMERA_SetMediaData(pMe->m_pICamera, &pMe->md , MT_JPEG);

// 2) Set the Size of the picture.
sz.cx = 320;
sz.cy = 240;
ICAMERA_SetSize(pMe->m_pICamera, &sz);

// 3) Adjust the Quality.
ADJUST_QUALITY(pMe, _CAM_MOV_HIGH);

// When needed set the Defer Encode mode. This will allow you
// to display the captured bitmap to the user before encoding it.
//ICAMERA_DeferEncode(pMe->m_pICamera, TRUE);

//Take a shot.
ICAMERA_Start(pMe->m_pICamera, CAM_MODE_SNAPSHOT, 0);
}
Since the Camera must be in the ready state when the snapshot mode has to be invoked, I recommend maintaining a applet level flag that is set when the user initiates the snapshot by pressing a key. Only the preview mode is stopped by the key press and the call to take the snapshot is made in the callback function when the CAM_STATUS_DONE for preview mode is returned.

Using the Defer Encode Mode:
The defer encode mode is a neat feature of the ICamera interface. It allows the developer to break down the taking picture operation into 2 distinct steps. When set, the picture taking operation stops after a frame has been captured by the camera. The applet can access the frame captured possibly to display a preview to the user or add some sort of graphics to it. To complete the picture taking operation, call the ICAMERA_EncodeSnapshot() method to finish the encoding.

To set the defer mode on, call ICAMERA_DeferEncode(pMe->m_pICamera, TRUE) method before invoking the snapshot. The captured frame can be accessed in the Callback function by calling the ICAMERA_GetFrame() function.

I am uploading a zip file containing the source code of the application for taking pictures and storing them in the applet directory on the device.Download Sample Application 2

Handling Suspend and Resume:
A brew Applet maybe suspended any time due an incoming call or ... Its completely upto the developer how he or she wants to handle this. Personally, I release the instance of ICamera when my applet is suspended and create it back again when the applet is resumed. Using the Pause mode of ICamera is another possibility however, I had some disturbing experiences with the pause mode earlier on and then never experimented.

One thing to keep in mind is In-comming calls while your applet has the ICamera interface in preview mode. The ringtone of the device does get distorted. I guess this has something to with the execution queue but not sure why. I have a Post on this that you can refer to for more details. I am looking for a better way to handle this, if you have any suggestions, please post them in the comments.

Movie Mode:
To be honest, haven't really found a phone that supports the movie mode. Its been some time since I actually was actively looking. If you know any device that allows a BREW applet to execute ICamera in movie mode then please do post it in the comments section of the post.

ICamera on the BREW Emulator:
The BREW SDK 2.1 emulators return EUNSUPPORTED while instantiating the ICamera interface. I did hear from someone that the 3.1 SDK's supported ICamera interface on the emulators using a webcam. Not sure about this. If some one did get this to work, then please do post your experience in the comments section.

Thursday, August 24, 2006

: Windows Mobile Product Documentation

Windows Mobile Product Documentation

Geez ... developing for Microsoft Sucks... Too many SDK's and not enough devices.

Thursday, August 10, 2006

DeviceAnywhere - Any device, Any network, Any global location

Have been using this site for testing out our applications. Pretty Neat ! Did find out that the application was misbehaving on a lot of phones. The system is pretty cool, you are connected to a real device and I would like to see more emulators built in similar fashion down the road. Won't that be cool.

The site is not perfect, the Launch conductor can hang at times but i suspect thats more to do with my PC than the actual app itself. You also need to have a really good connection speed.

Positives:
1) As a developer I do not need to buy all the phones atleast not the crappy ones.
2) Gives the QA the ability to record frames so that there are no more "Cannot reproduce" statements in the test report.

Negatives:
1) I do not think their implementation is perfect for all devices. I have had some issues with screen refreshing that I do not think I am responsibloe for specially since they do not occur on the live device.
2) Would like to see more devices added to their list [OK ... this is not a negative]

They have a special promotion to try out the Beta product for 199.99 per month. I do not know how different this is from the actual product for 399.99 per month. I have been using the Beta product.

Hopefully, sites like Device Anywhere will drop their prices in the near future once they have made up their initial investement so that more developers can get access to devices.

DeviceAnywhere - Any device, Any network, Any global location

Sunday, August 06, 2006

BlackBerry 8100 cameraphone! Stealth revealed! - Engadget

Just saw this at Engadget. Looks awesome. Cant make out if it has a Scroll wheel though.

BlackBerry 8100 cameraphone! Stealth revealed! - Engadget

Revisiting ScanR

Its been a while since I checked out the ScanR application. They have made some nice changes. They now have a seperate address for sending documents and whiteboards. I guess its just some time before they enter into the product matching space. Love the google gadget they have come up with. Added it to my home page instantly. I guess technically speaking it should be simple to implement a google gadget. I am now adding it to my TODO list ;-).

My Earlier Post on ScanR
ScanR Blog

Friday, August 04, 2006

[WAP] SCANBUY Shoppers WAP site is now online

The beta version of SCANBUY Shopper is now finally up and running. The WAP site is now available at http://search.scanbuy.com