📸 MacOS Media Capture with CoreMediaIO: In-Depth Summary & Developer Guide

Primary Technologies: CoreMediaIO, C++, Objective-C, AVFoundation, macOS

---

🎯 What This Is About

macOS provides various APIs to access camera, audio, and screen capture input. While AVFoundation is typically used for such tasks, it doesn’t provide access to raw or compressed (muxed) media buffers. This limitation is critical when:

• You’re working with screen mirroring from iOS devices.
• You need to access compressed or encoded data (H.264 video, AAC audio, etc.).
• You require deeper control over low-level capture streams, like from Blackmagic or Elgato capture cards.

To overcome this, CoreMediaIO (CMIO) is used—a lower-level framework in the AV ecosystem that allows direct hardware interaction and media data capture.

---

💡 Why Use CoreMediaIO?

Feature	AVFoundation	CoreMediaIO
High-level API	✅ Yes	❌ No
Access to raw hardware stream	❌ No (decoded only)	✅ Yes (raw/muxed supported)
Compressed/muxed data	❌ No	✅ Yes
Screen mirroring capture	❌ Limited	✅ Supported
DAL device control	❌ No	✅ Yes

If you need to interact with a device as it presents data—without pre-processing—you need CoreMediaIO.

---

🔧 Step-by-Step Implementation Breakdown

✅ 1. Enable DAL Devices (iOS Screen Mirror Capture)

To make devices like iOS screen mirroring appear to the system, enable them manually using:

void EnableDALDevices() {
    CMIOObjectPropertyAddress prop = {
        kCMIOHardwarePropertyAllowScreenCaptureDevices,
        kCMIOObjectPropertyScopeGlobal,
        kCMIOObjectPropertyElementMaster
    };
    UInt32 allow = 1;
    CMIOObjectSetPropertyData(kCMIOObjectSystemObject, &prop, 0, NULL, sizeof(allow), &allow);
}

🔸 Use case: Enables hidden input sources like iOS devices, which normally won’t appear under AVCaptureDevice.

---

✅ 2. Monitor Device Add/Remove Events

You can track when devices are connected or disconnected using:

NSNotificationCenter *notiCenter = [NSNotificationCenter defaultCenter];
id connObs = [notiCenter addObserverForName:AVCaptureDeviceWasConnectedNotification
                                     object:nil queue:[NSOperationQueue mainQueue]
                                 usingBlock:^(NSNotification *note) {
                                     AVCaptureDevice *device = note.object;
                                     NSLog(@"New device connected: %@", device.localizedName);
                                 }];

🔸 Real-time device monitoring—great for dynamic apps like streaming or diagnostics tools.

---

✅ 3. List Devices (AVFoundation Layer)

Use this for quick debugging or for cross-referencing UIDs.

NSArray *devs = [AVCaptureDevice devices];
for (AVCaptureDevice* d in devs) {
    NSLog(@"uniqueID: %@\n", [d uniqueID]);
    NSLog(@"modelID: %@\n", [d modelID]);
    NSLog(@"description: %@\n", [d localizedName]);
}

---

✅ 4. Resolve CoreMediaIO Device by Unique ID

CoreMediaIO doesn’t use the same identifiers as AVFoundation, so a bridge is required. Example signature from the article:

OSStatus FindDeviceByUniqueId(const char* pUID, CMIODeviceID& devId);

🔸 This maps an AVFoundation UID (NSString) to a CoreMediaIO device ID, enabling full CMIO-level access.

---

📦 Core Utility Functions Explained

Here are some reusable tools described in the original article:

Function Name	Purpose
GetNumberDevices()	Get total number of CMIO devices.
GetDevices()	Fetch list of all CMIODeviceIDs.
GetDeviceStrProp()	Extract string properties from a CMIO device (e.g., UID).
CFStringCopyUTF8String	Convert CFStringRef to UTF-8 (important for C++ interoperability).
GetPropertyData()	Universal getter for property values (variant types).
GetPropertyDataSize()	Gets expected buffer size for CMIO property queries.

---

📡 Missing But Essential: Stream Handling & Capture

The article stops short of actual data capture, which would involve:

1. Enumerating Streams:
   • Each device has one or more streams (CMIOStreamID).
2. Stream Format Negotiation:
   • Query stream properties to determine format (e.g., CMVideoFormatDescription).
3. Registering Listeners:
   • Use CMIOStreamCopyBufferQueue or hardware device listeners to receive data buffers.

This would complete the loop to process raw H.264, AAC, or even raw screen video.

---

⚠️ Developer Notes & Warnings

• 💥 CoreMediaIO is poorly documented, and very few Apple sample projects use it.
• 💡 Understanding C++ and Objective-C interoperability is essential.
• 🔐 Privacy permissions must be granted manually for camera/screen access in System Preferences > Security & Privacy.
• 🧪 Many useful tools (like DALPlugInTester) are part of Apple’s internal toolchain only.

---

🚀 Real-World Use Cases

• Live Streaming Software (OBS-like apps)
• iOS Screen Mirroring Tools
• Custom Video Capture Utilities
• Muxed Stream Recording (e.g., H.264 streams from HDMI capture cards)
• Professional Broadcasting Apps