A camera device on Android is special in many ways. A camera device is a Conceptually, a camera device produces a continuous stream of data from its image sensor. Unlike a “byte stream” device, however, this data from a camera device requires interpretation based on image serialization formats. A camera device can be seen as a “source” device.
The data from a camera device is consumed by one or more “sink” mechanisms. Each “sink” mechanism taps into the source possibly in its own way that is different from the other mechanisms. Each “sink” listens to the source and manages its own way to capture data from the “sink” in a manner that can be utilized by an app.
A camera device cannot be connected to multiple processes at the same time, and as a result its management requires some special consideration.
Start with the following lines in the AndroidManifest.xml file:
This line should be placed after the <application> element. This line informs older versions of Android systems that permission to the camera(s) is requested.
As per the Android Developer document that explains run-time permission requests, even with the <uses-permission> element in the AndroidManifest.xml file, an app should still request permission at run-time.
This complicates the run-time code a little because there are now two places to trigger the execution of the code when permission is granted. An app remembers whether a permission is already granted. If a permission is already granted as returned by ContextCompat.checkSelfPermission, then the app can proceed. Otherwise, an app needs to request permission using ActivityCompat.requestPermissions. The confirmation of the requested permission should be processed in a onRequestPermissionsResult method.
If an app needs to use camera devices, it needs the Manifest.permission.CAMERA permission.
In order to get a camera to work, many objects are needed. Some objects are only needed in a transient basis.
A CameraManager object represents an interface to the camera management of Android. As such, this object lets an app inquire what cameras are available as well as the capabilities of each camera.
A CameraCharacteristics object is obtained from calling the getCameraCharacteristics method of a CameraManager object. A CameraCharacteristics object represents the characteristics of a specific camera, each one represented by a key-value pair. By using a key-value mechanism and a flexible return type, the get method has a lot of flexibility and potential to change in the future without any actual Java code change.
A StreamConfigurationMap object is one of the many values of a CameraCharacteristics object using the key of CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP. A StreamConfigurationMap object is another level of configuration details. However, this one is specific to the “stream” of a camera device. Of particular interest in the context of this module is the method getOutputFormats. The return value of getOutputFormats is an array of Size objects. Each one represents a particular native size of the stream from a specific camera device. It is important to choose one of these Size objects when we configure the “sink” mechanism.
A CameraDevice represent a specific camera on an Android device. The request to open a camera is triggered by calling openCamera from a CameraManager object. However, this call does not return with a reference to a CameraDevice object. Instead, the call returns immediately. A callback object is passed to openCamera to actually handle the case when a camera is connected using the onOpened method of a CameraDevice.StateCallBack object.
On the “sink” side, a SurfaceView UI element can be used to preview what a camera is seeing. An Activity can include a SurfaceView in its layout. However, the actual size of the SurfaceView element should be determined at run-time. Use “wrap-content” as the width and height.
Unlike most other views, a SurfaceView element has a Surface object in it. The creation of a Surface object is time and resource consuming. As a result, it is not created on-the-fly. Furthermore, a SurfaceHolder object is associated with a SurfaceView object to control the underlying Surface object of a SurfaceView element.
The main reason a SurfaceHolder interface is created instead of integrating it into SurfaceView is not to complicate SurfaceView with methods that are specific to handling a Surface.
Note that whatever logic that needs access to a Surface should attach to the surfaceCreated method of a SurfaceHolder.Callback interface. The reference of an object that implements SurfaceHolder.Callback should be registered using the addCallback method of the SurfaceHolder object associated with a SurfaceView.
Once a Surface is created, its size should be configured using the setFixedSize method of the SurfaceHolder that is containing it. This size should be one of the Sizes supported by a camera device as discussed earlier.
A Surface object represents the low-level “frame-buffer”. A “frame-buffer” is the interface between the special camera-dependent source of video data and the image representation that is accessible to an app. A Surface object is not the only way to capture a frame from the video stream of a camera.
A CameraCaptureSession connects a CameraDevice object (the camera behind it) to a Surface object. The createCaptureSession method of a CameraDevice object attempts to make this connection. Again, the connection is not made when the method returns. Instead, a callback object is passed to specify what to do based on the result of the request. The onConfigured method of the interface CameraCaptureSession.StateCallback specifies the logic to execute when a session is configured.
When a session is configured, a CameraCaptureSession object is passed. This object represents the connection between the continuous output stream of a camera device and a frame-buffer that is accessible to an app. However, this connection does not actually do anything until a capture is requested. This applies even to previewing what a camera is seeing.
A CameraRequest.Builder object is returned by calling the createCaptureRequest method of a CameraDevice object. A CameraRequest.Builder object is associated with a template for specific ways to capture a frame. A builder object needs to be associated with at least one target of the capture. A target of capture is a frame-buffer, a Surface object.
Once a builder object is fully configured, it builds an actual request using the method build. This request is then passed to a CameraCaptureSession object using its setRepeatingRequest method (or other set...Request method).
A set...Request method of a CameraCaptureSession object can optionally take a callback object that specifies what to do when a frame is captured.
Any app that makes use of cameras has a lot of callbacks. This is because many operations are performed by the operating system and not utilizing the main/UI thread of an app process.
The use of callbacks is a way for an app to “hand off” long and complex OS operations to Android. Most of the time, there is little or nothing to do after calling a method that has a callback because the result of the method is not known until one of the methods of the callback object is called.
When Android is done with a request that has a callback object, Android places the callback object in the main message queue of the requesting app including information about which method of the callback object should be used and how its parameters should be specified. The looper of the main/UI thread then picks up the callback object (and extra information) and performs the specified method call.
Note that calls to methods containing callback objects can be done in threads other than the main/UI thread. However, any code that handles UI related objects should still be performed in the main/UI thread. Calls to callback object methods are always performed in the same thread as the one that made the request to begin with.
An image can be acquired using a process similar to what is described in details. An ImageReader object has an associated Surface object. This Surface object can be used in capture requests already described in an earlier section. The only difference is that in the case of capturing an image (as opposed to previewing), the image data can be processed and/or saved.