Qt QML Camera to C++ QImage on Android

1) Yes, it is possible. I´ve come around two ways to do it.

Using QAbstractVideoFilter alongside with QVideoFilterRunnable classes (QT 5.5 only! ) which are plain great. They were developed specifically for this case scenario, and are pretty easy to use.

There are a few good examples on the web using it:

https://blog.qt.io/blog/2015/03/20/introducing-video-filters-in-qt-multimedia/

http://code.qt.io/cgit/qt/qtmultimedia.git/tree/examples/multimedia/video/qmlvideofilter_opencl

The downside to this approach is, like was said here , is that on Android devices the QVideoFrame pointer doesnt has raw pixel data, instead, it has an OpenGL Texture which need to be read back (the second example I posted has a workaround solving this), thus making this approach not realy good for real time purposes IMHO.

What I ended up using to solve this problem was the QVideoProbe class.

First you have to name the instance of your QML camera:

    Camera {    id: camera    objectName: "qrCameraQML"}

Then you get this instance from C++ side, something like:

QObject *qmlCamera = engine.rootObjects().at(0).findChild<QObject*>("qrCameraQML");

The QML camera instance actually has a QVariant element accessible only via C++ that can be cast into a QCamera* :

camera_ = qvariant_cast<QCamera*>(qmlCamera->property("mediaObject"));

Then all you have to do is to connect the probe to a slot that will actually handle the QVideoFrames and then set the source of the probe as the QCamera* previouslly cast:

    connect(&probe_,SIGNAL(videoFrameProbed(QVideoFrame)),this,SLOT(handleFrame(QVideoFrame)));probe_.setSource(camera_);

On my example camera_ and probe_ are simply:

    QCamera *camera_;QVideoProbe probe_;

This approach on my experience was a lot faster(for android platforms) than using the qt video filter classes, but it has the disadvantage that you basically only read the video output from qml, and AFAIK you wont be able to send postprocessed videoframes back to qml.

If you really need to send the processed images back to qml i'd advise you to try the first approach and see what happens.

2)Not with Qt AFAIK, maybe with OpenCv, or some other lib.

I like to highlight @waldez-junior first answer. In QML, you add your QAbstractVideoFilter component to the VideoOutput.

Camera {    id: camera}VideoOutput {    id: videoOutput    source: camera    filters: [ videoFilter ]    autoOrientation: true}MyVideoFilter {    id: videoFilter    // orientation: videoOutput.orientation}

In C++, you implement QAbstractVideoFilter component, here's a minimal example:

class MyVideoFilter : public QAbstractVideoFilter{    Q_OBJECTpublic:    QVideoFilterRunnable *createFilterRunnable() Q_DECL_OVERRIDE    {        return new CustomFilterRunnable(this);    }};class MyVideoFilterRunnable : public QVideoFilterRunnable{public:    QVideoFrame run(QVideoFrame* input, const QVideoSurfaceFormat& surfaceFormat, RunFlags flags)    {        if (!input->isValid())        {            return *input;        }        // do stuff with input        return *input;    }};

```

There's an QAbstractVideoFilter example in Qt source code: http://code.qt.io/cgit/qt/qtmultimedia.git/tree/examples/multimedia/video/qmlvideofilter_opencl.

To make things easier, consider using the Qt internal function qt_imageFromVideoFrame for converting a QVideoFrame to a QImage. This code works for NoHandle case and works on most platforms. Unfortunately, it doesn't work for at lot of Android devices because QVideoFrame::map() will return false.

extern QImage qt_imageFromVideoFrame(const QVideoFrame& f);

For Android, you need to handle the GLTextureHandle case where you use OpenGL to populate a QImage.

On some devices, the image internal bit buffers will appear flipped.

#ifdef Q_OS_ANDROID    bool flip = true;#else    bool flip = surfaceFormat.scanLineDirection() == QVideoSurfaceFormat::BottomToTop;#endif

On some devices, the image may also be rotated. The best way for dealing with rotation is set autoOrientation: true in the VideoOutput component. Then, your component can just take a copy of videoOutput.orientation.

Knowing how an image is flipped and rotated will assist in video recognition applciations (e.g. face recognition).

I've also created a minimum working sample on https://github.com/stephenquan/MyVideoFilterApp

1. I think the answer above sufficiently explained processing from the QML Camera
2. Yes, there are other possibilities I found this project helped me a lot:https://github.com/rferrazz/CvCamViewregister the plugin to QML and it can be used like this:

import QtQuick 2.3import QtQuick.Window 2.2import CVComponents 1.0Window {    visible: true    CVCAM{        id: camera        width: 640        height: 480        deviceId: "0"        imageHeight: 640        imageWidth: 480    }}

Processing the image is as very simple. The class only paints the item so it can be used on QML all other processing goes on in the back end, so the Mat image used by the camera can be used for processing.