BREW: ICamera Interface

I have been meaning to write this post for quiet some time now and finally got around to finishing it. I am including a few tid bits in this post about the BREW ICamera Interface. The post is not meant to be a tutorial but could provide you a starting block if you are looking to add the camera functionality into your BREW Applet and then some.
For tutorials, I recommend the following:

• Devx:Camera-enable Your Applications with BREW's ICamera APIs by Ray Rischpater.
  A nice short tutorial to get you started with the ICamera Interface. In case the above link is broken, try searching on Google or Devx.com for the article.
  
• BREW forums
  If the above link does not work, search for ICamera and you will get plenty of posts with issues about the interface and also notes on how to solve them.

Ok ... Now that you have the links, lets check out the ICamera interface.

Determine Device Support:
The ICamera interface was introduced in BREW version 2.1. The support for the interface was slow initially but now pretty much any camera phone coming out supports the interface. To check for support before actually buying the device, take a look at the BREW specs (Data Sheet) for the device available through the Qualcomm BREW developer website. Usually there is a section called "Camera" in the spec sheet dedicated to camera access that provides information about:

• Camera Available to BREW Applet.
• Resolution (usually the native camera resolution).
• Access to native photo directory.
• Path of the native photo directory.

Creating the ICamera Interface Instance:
Let's start with the header files:

• AEECamera.h: Provides the definitions of the ICamera interface and will have to be included in your build.
• AEEBitmap.h: Provides the definitions of the IBitmap interface. The frame captured by the camera is returned as IBitmap, so this is important.
• AEEMimeTypes.h: Contains the definitions of the mime types etc and you might need this if you are planning on taking snapshots etc...

To create an Instance of ICamera, we resort to the trusty ISHELL interface.

ICamera *m_pICamera = NULL; int nRet = EBADPARM; nRet = ISHELL_CreateInstance(pMe->iShell, AEECLSID_CAMERA, (void **)&m_pICamera);

If access to camera is supported and an instance was instantiated, the function returns SUCCESS. Other common codes returned include:

• EBADPARM: Check the pointers you are passing to the function.
• EUNSUPPORTED: You should really check the specs on the device before trying to instantiate the ICamera interface.
• EPRIVLEVEL: This is weird but happens on certain devices. To avoid getting this return code, you can try adding a dependency for the camera in your MIF. More details on this here.

Once we have our instance of ICamera, the next thing to do would be to register a Callback function. All a developer has to do is to have a function that corresponds to the PFNCAMERANOTIFY specification.

typedef void (*PFNCAMERANOTIFY)(void* pUser, AEECameraNotify * pNotify);
The implementation of ICamera interface is Asynchronous similar to the other interfaces like IMedia and IWeb. The operation of the camera and the access to it revolves around the BREW layer notifying the Applet via the callback function. More information on this can be found in the BREW SDK documentation.The RegisterNotify function is used to register the Callback function.

/*Registering the callback function. m_pICamera: Instance of ICamera created above _CameraNotify: Callback Function pMe: Pointer to the instance of the Applet. You can pass anything here. This ptr will be returned in the callback as the void pointer pUser. */ nRet = ICAMERA_RegisterNotify(m_pCamera, _CameraNotify, pMe);

For more details on the AEECameraNotify structure, refer to the BREW SDK documentation. I usually use the following members of the structure to determine the corresponding action in the callback function.
nStatus : The current status.
nCmd and nSubcmd: Gives an indication of the current camera mode.

//Generic Implementation of the callback function.void _CameraNotify(void * pUser, AEECameraNotify * pn) { //Get a handle to the reference pointer. MyObj* pMe = (MyObj *) pUser; if(!pMe !pn) return; //Handling based on Status passed. switch(pn->nStatus) { case CAM_STATUS_START: /*Sent when Preview or Record operation is started.*/ if(pn->nSubCmd == CAM_MODE_PREVIEW && pn->nCmd == CAM_CMD_START) { /*preview Mode has started.*/ } if(pn->nSubCmd == CAM_MODE_SNAPSHOT &&pn->nCmd == CAM_CMD_START) { /* This is the first callback recieved when you initiate the Snapshot mode.*/ } break; case CAM_STATUS_DONE: /* [Preview/Record/SetParm/GetParm/EncodeSnapshot] Operation completed successfully. For RecordSnapShot, pData = TRUE/FALSE = > Defered encode enabled/disabled */ if(pn->nSubCmd == CAM_MODE_PREVIEW && pn->nCmd == CAM_CMD_START) { /*Preview Mode has completed.*/ } else if(pn->nSubCmd == CAM_MODE_SNAPSHOT && pn->nCmd == CAM_CMD_START) { /* Snapshot has been taken. In defer mode, the bitmap captured can be accessed here. In default mode, you can wait for the snapshot encoding to finish. if(pn->nSubcmd = = CAM_CMD_ENCODESNAPSHOT)*/ } if(pn->nCmd == CAM_CMD_SETPARM) { /* When a parm such as Zoom etc has been set.*/ } break; case CAM_STATUS_FAIL: /* [Preview/Record/SetParm/GetParm/EncodeSnapshot] Operation failed pData = CAM_EXXX error code. You can use the pn->nCmd andpn->nSubCmd to see exactly what operation failed. */ break; case CAM_STATUS_ABORT: /*The last operation was aborted. The camera is now in the ready state.*/ break; case CAM_STATUS_FRAME: /* [Any] Frame captured by camera. The frame can be accessed using ICAMERA_GetFrame()*/ break; case CAM_STATUS_PAUSE: /* [Preview/Record] Record movie paused. This status is returned when you call ICAMERA_Pause()*/ break; case CAM_STATUS_RESUME: /* [Preview/Record] Record movie resumed. AEECameraNotify::pData is IBitmap pointer representing the snapshot*/ break; case CAM_STATUS_DATA_IO_DELAY: /* [Preview/Record] Operation being delayed by data i/o access*/ break; case CAM_STATUS_SPACE_WARNING: /* [Record] Memory available to store recording running low. To Do: Check if this applies to Snapshots too. Havent really checked this yet.*/ break; } }

Now that we have the different status codes that are returned in the Callback, lets check out the modes.

Starting the Camera (PREVIEW Mode)
J2ME developers will find the approach a bit different, there is NO Videocontrol :-). The developer is responsible for getting the captured frame from the instance and then blitting it on the LCD. The approach is very similar to the Series 60 API's and maybe microsoft smart phones (I am not sure).
Things to do:

• Set the Size of the Preview Frame.
• Start the Preview Mode.

Offcourse, there are bunch of Optional things you can do before starting the preview mode but remember, just cause its stated in the API, doesn't mean its supported on the phone.

The BREW API provides the developer a way to query the supported resolution for rendering the preview frames. Again, not all phones will support this method. The simplest way to set the size for the preview frames is to use the device LCD size. Since the frame has to be rendered on the device screen, might as well use this size(Japanese phones from KDDI do provide a preview frame of size larger than the LCD. But the frame is scaled when you blit it on the entire screen.). Using the device screen size gives the default preview size for the device.

static void initDisplaySizes(cameratest * pMe) { AEESize *ltSize; int loop; boolean bRet= FALSE; int nRet = EFAILED; //Get the Resolution available for the Preview Mode. ltSize = (AEESize *)CAM_MODE_PREVIEW; nRet = ICAMERA_GetDisplaySizeList(pMe->m_pICamera, if(SUCCESS != nRet) { // There was an error in getting the sizes. use the screen size. return; } // Got the available sizes. // Run through the loop and get the // best resolution possible. for(loop = 0 ; ltSize[loop].cx != NULL && ltSize[loop].cy != NULL ; loop++) { // Width: ltSize[loop].cx // Height: ltSize[loop].cy } }

To start capturing and displaying the frames:

static void cameratest_DisplayCameraPreview(cameratest * pMe) { int nRet; AEESize az; // Setting the Display Size. Using Screen Size. az.cx = pMe->cxScreen; az.cy = pMe->cyScreen; ICAMERA_SetDisplaySize(pMe->m_pICamera, &az); // Start the Preview mode. // This call will start the messages to be // sent to the callback function. nRet = ICAMERA_Preview(pMe->m_pICamera); }

Once the Preview mode has started, a notification with the status CAM_STATUS_START is sent to the callback function. In case of error, a corresponding error status is sent. The individual frames captured by the camera are sent with the status CAM_STATUS_FRAME. The developer must implement logic here to display the frames on the screen.

void _CameraNotify(void *pUser, AEECameraNotify *pn) { cameratest * pMe = (cameratest *) pUser; if(!pMe !pn) return; switch(pn->nStatus) { case CAM_STATUS_START: //Preview mode has started. Do any applet specific // stuff here. break; case CAM_STATUS_FRAME: // A frame has been captured by the camera and // is now available for rendering on the LCD. IBitmap * pFrame; AEEBitmapInfo bi; //Get the captured frame ICAMERA_GetFrame(pMe->m_pICamera, &pFrame); if (!pFrame) break; //Error //Get the bitmap info... //IBITMAP_GetInfo(pFrame, &bi, sizeof(bi)); //Blitting the frame on theLCD from (0,0) IDISPLAY_BitBlt(pMe->pIDisplay,0, 0,pMe->cxScreen, pMe->cyScreen, pFrame, 0, 0, AEE_RO_COPY); // Release the Bitmap instance IBITMAP_Release(pFrame); //Update the Display IDISPLAY_Update(pMe->pIDisplay); break; case CAM_STATUS_DONE: // The camera has stopped the preview mode // and has entered the ready state. break; } }

To stop the camera in preview mode, use the ICAMERA_Stop() function. This will send the CAM_STATUS_DONE status to the callback function.

Adjusting Parameters:
Now that the camera is running in the preview mode, lets take a look at some of the control parameters available to the developer. Digital Zoom, Brightness, Contrast etc. Are some of the control parameters that are available through the BREW api. For the complete list check the SDK documentation. The SDK provides wrapper functions for adjusting these parameters and also checking if they are supported (ICAMERA_IsBrightness). I personally prefer using the ICAMERA_SetParm and ICAMERA_GetParm methods eventhough I am probably replicating code :-).

#define _CAM_MOV_UP 1 // Move one Step Up #define _CAM_MOV_DOWN 2 // Move one Step down #define _CAM_MOV_HIGH 4 // Move to highest #define _CAM_MOV_LOW 8 // Move to the lowest #define ADJUST_QUALITY(po, pDir) adjustParm(po, pDir,CAM_PARM_QUALITY) #define ADJUST_ZOOM(po, pDir) adjustParm(po, pDir,CAM_PARM_ZOOM) #define ADJUST_BRIGHTNESS(po, pDir) adjustParm(po, pDir,CAM_PARM_BRIGHTNESS) #define ADJUST_CONTRAST(po, pDir) adjustParm(po, pDir,CAM_PARM_CONTRAST) #define ADJUST_SHARPNESS(po, pDir) adjustParm(po, pDir,CAM_PARM_SHARPNESS) static int adjustParm(cameratest * pMe, int pDirection, int parmID) { int nRet; AEEParmInfo pi; int32 p1; int adj = 0; //Adjusted Value //Get the Parm Value nRet = ICAMERA_GetParm(pMe->m_pCamera,(int16) parmID, &p1,(int32 *)&pi); /**@TODO: Handle CAM_PENDING */ if(SUCCESS = = nRet) { //Calculate the next step switch(pDirection) { case _CAM_MOV_UP: adj = pi.nCurrent + pi.nStep; break; case _CAM_MOV_DOWN: adj = pi.nCurrent - pi.nStep; break; case _CAM_MOV_HIGH: adj = pi.nMax; break; case _CAM_MOV_LOW: adj = pi.nMin; break; default: return EUNSUPPORTED; } //Set the New Value nRet = ICAMERA_SetParm(pMe->m_pCamera,(int16) parmID,(int32) adj, 0); } return nRet; }

To increase the digital zoom all I have to do is call ADJUST_ZOOM macro. I am uploading a zip file containing the source code of the application for the preview mode and also the ARM binaries. My development environment is Microsoft Windows .NET 2003. In case the link below does not work, shoot me an email and I will send you the zip file.
Download Sample Application 1

Taking Pictures:
The most obvious use of the ICAMERA interface in any applet is to take snapshots. The SNAPSHOT mode of the interface allows you to do this. One thing to remember is that the ICamera instance MUST be in the READY state for the operation to succeed.
Things to do:

• Make sure the camera is in READY state. If you are in preview mode then call the ICAMERA_Stop() function to get there.
• Set the Media data (ICAMERA_SetMediaData()).
• Set the size of the picture to be taken (ICAMERA_SetSize()).
• Set the encoding for the image (ICAMERA_SetVideoEncode()).
• Adjust the quality of the picture to be taken (ICAMERA_SetQuality()).
• Initiate the picture taking operation (ICAMERA_RecordSnapshot()).

Taking a snapshot is a 2 stage operation, the first step is the actual capturing of the frame and the second step deals with encoding the snapshot. Another mode, ENCODESNAPSHOT maintains the camera state when the encoding is being done.

static void takePicture(cameratest * pMe) { AEESize sz; // 1) Set Media Data pMe->md.clsData = MMD_FILE_NAME; pMe->md.pData = "snap.jpeg" ; //Store the picture in the applet directory pMe->md.dwSize = 0 ; ICAMERA_SetMediaData(pMe->m_pICamera, &pMe->md , MT_JPEG); // 2) Set the Size of the picture. sz.cx = 320; sz.cy = 240; ICAMERA_SetSize(pMe->m_pICamera, &sz); // 3) Adjust the Quality. ADJUST_QUALITY(pMe, _CAM_MOV_HIGH); // When needed set the Defer Encode mode. This will allow you // to display the captured bitmap to the user before encoding it. //ICAMERA_DeferEncode(pMe->m_pICamera, TRUE); //Take a shot. ICAMERA_Start(pMe->m_pICamera, CAM_MODE_SNAPSHOT, 0); }

Since the Camera must be in the ready state when the snapshot mode has to be invoked, I recommend maintaining a applet level flag that is set when the user initiates the snapshot by pressing a key. Only the preview mode is stopped by the key press and the call to take the snapshot is made in the callback function when the CAM_STATUS_DONE for preview mode is returned.

Using the Defer Encode Mode:
The defer encode mode is a neat feature of the ICamera interface. It allows the developer to break down the taking picture operation into 2 distinct steps. When set, the picture taking operation stops after a frame has been captured by the camera. The applet can access the frame captured possibly to display a preview to the user or add some sort of graphics to it. To complete the picture taking operation, call the ICAMERA_EncodeSnapshot() method to finish the encoding.

To set the defer mode on, call ICAMERA_DeferEncode(pMe->m_pICamera, TRUE) method before invoking the snapshot. The captured frame can be accessed in the Callback function by calling the ICAMERA_GetFrame() function.

I am uploading a zip file containing the source code of the application for taking pictures and storing them in the applet directory on the device.Download Sample Application 2

Handling Suspend and Resume:
A brew Applet maybe suspended any time due an incoming call or ... Its completely upto the developer how he or she wants to handle this. Personally, I release the instance of ICamera when my applet is suspended and create it back again when the applet is resumed. Using the Pause mode of ICamera is another possibility however, I had some disturbing experiences with the pause mode earlier on and then never experimented.

One thing to keep in mind is In-comming calls while your applet has the ICamera interface in preview mode. The ringtone of the device does get distorted. I guess this has something to with the execution queue but not sure why. I have a Post on this that you can refer to for more details. I am looking for a better way to handle this, if you have any suggestions, please post them in the comments.

Movie Mode:
To be honest, haven't really found a phone that supports the movie mode. Its been some time since I actually was actively looking. If you know any device that allows a BREW applet to execute ICamera in movie mode then please do post it in the comments section of the post.

ICamera on the BREW Emulator:
The BREW SDK 2.1 emulators return EUNSUPPORTED while instantiating the ICamera interface. I did hear from someone that the 3.1 SDK's supported ICamera interface on the emulators using a webcam. Not sure about this. If some one did get this to work, then please do post your experience in the comments section.