flutter real time face detection
For the real-time access to camera image stream, I answered in another question How to access camera frames in flutter quickly that you want to use CameraController#startImageStream
import 'package:camera/camera.dart';import 'package:flutter/foundation.dart';import 'package:flutter/material.dart';void main() => runApp(MaterialApp(home: _MyHomePage()));class _MyHomePage extends StatefulWidget { @override _MyHomePageState createState() => _MyHomePageState();}class _MyHomePageState extends State<_MyHomePage> { dynamic _scanResults; CameraController _camera; bool _isDetecting = false; CameraLensDirection _direction = CameraLensDirection.back; @override void initState() { super.initState(); _initializeCamera(); } Future<CameraDescription> _getCamera(CameraLensDirection dir) async { return await availableCameras().then( (List<CameraDescription> cameras) => cameras.firstWhere( (CameraDescription camera) => camera.lensDirection == dir, ), ); } void _initializeCamera() async { _camera = CameraController( await _getCamera(_direction), defaultTargetPlatform == TargetPlatform.iOS ? ResolutionPreset.low : ResolutionPreset.medium, ); await _camera.initialize(); _camera.startImageStream((CameraImage image) { if (_isDetecting) return; _isDetecting = true; try { // await doOpenCVDectionHere(image) } catch (e) { // await handleExepction(e) } finally { _isDetecting = false; } }); } Widget build(BuildContext context) { return null; }}
I did something this with OpenCV before, my solution was:
- Start a new Activity or ViewController on Android and iOS respectively via platform channel. Example:
class FaceScanPlugin(val activity: Activity) : MethodCallHandler, PluginRegistry.ActivityResultListener {
var result: Result? = nullcompanion object { @JvmStatic fun registerWith(registrar: Registrar): Unit { val channel = MethodChannel(registrar.messenger(), "com.example.facescan") val plugin = BarcodeScanPlugin(registrar.activity()) channel.setMethodCallHandler(plugin) registrar.addActivityResultListener(plugin) }}override fun onMethodCall(call: MethodCall, result: Result): Unit { if (call.method.equals("scan")) { this.result = result showFaceScanView() } else { result.notImplemented() }}private fun showFaceScanView() { val intent = Intent(activity, FaceScannerActivity::class.java) activity.startActivityForResult(intent, 100)}override fun onActivityResult(code: Int, resultCode: Int, data: Intent?): Boolean { if (code == 100) { if (resultCode == Activity.RESULT_OK) { return true } } return false}}
Refer to Flutter QR scanner plugin on how to navigate to Android activity or iOS View.
- Then do your OpenCV real-time face detection via Camera2 and AVFoundation.
Other than that, I supposed you can try out the new AndroidView or UIKitView if you want to embed the android or iOS into your Flutter app.