Commit 9877c92
committed
Add support for RTCRtpScriptTransform
https://bugs.webkit.org/show_bug.cgi?id=219148
Reviewed by Eric Carlson.
Source/WebCore:
We introduce RTCRtpScriptTransform which processes encoded frames in a worker for either RTCRtpSender or RTCRtpReceiver.
The model follows AudioWorkletNode in the sense that we create a RTCRtpScriptTransform object in main thread that is used with RTCRtp objects.
The RTCRtpScriptTransform takes a name and a worker as parameters to create a RTCRtpScriptTransformer counter part in a worker.
Before that, RTCRtpScriptTransformer constructors are registered in the worker with a specific name.
A message port is shared between RTCRtpScriptTransform and RTCRtpScriptTransformer.
RTCRtpScriptTransform keeps a weak pointer to RTCRtpScriptTransformer so that we keep all ref counting of RTCRtpScriptTransformer in the worker thread.
To make sure RTCRtpScriptTransformer stays alive for long enough, we set a pending activity when RTCRtpScriptTransform is linked to its RTCRtpScriptTransformer.
The pending activity is then removed either at worker closure or RTCRtpScriptTransform being no longer doing processing.
We expose individual compressed frames as RTCEncodedAudioFrame and RTCEncodedVideoFrame.
Accessor is limited to the raw data but additional getters should be added later on.
To implement RTCRtpScriptTransformer, we have to be able to create WritableStream with native sinks.
This is why we introduce WritableStreamSink and WritableStream C++ classes.
Binding between native frames and streams is done through RTCRtpReadableStreamSource and RTCRtpWritableStreamSink.
Test: http/wpt/webrtc/webrtc-transform.html and http/wpt/webrtc/sframe-transform.html.
* CMakeLists.txt:
* DerivedSources-input.xcfilelist:
* DerivedSources-output.xcfilelist:
* DerivedSources.make:
* Modules/mediastream/RTCEncodedAudioFrame.cpp: Added.
(WebCore::RTCEncodedAudioFrame::RTCEncodedAudioFrame):
* Modules/mediastream/RTCEncodedAudioFrame.h: Added.
(WebCore::RTCEncodedAudioFrame::create):
* Modules/mediastream/RTCEncodedAudioFrame.idl: Added.
* Modules/mediastream/RTCEncodedFrame.h: Added.
* Modules/mediastream/RTCEncodedFrame.cpp: Added.
* Modules/mediastream/RTCEncodedVideoFrame.cpp: Added.
(WebCore::RTCEncodedVideoFrame::RTCEncodedVideoFrame):
* Modules/mediastream/RTCEncodedVideoFrame.h: Added.
(WebCore::RTCEncodedVideoFrame::create):
* Modules/mediastream/RTCEncodedVideoFrame.idl: Added.
* Modules/mediastream/RTCRtpReceiver+Transform.idl:
* Modules/mediastream/RTCRtpReceiver.cpp:
(WebCore::RTCRtpReceiver::setTransform):
(WebCore::RTCRtpReceiver::transform):
* Modules/mediastream/RTCRtpReceiver.h:
* Modules/mediastream/RTCRtpReceiverWithTransform.h:
(WebCore::RTCRtpReceiverWithTransform::transform):
(WebCore::RTCRtpReceiverWithTransform::setTransform):
* Modules/mediastream/RTCRtpSFrameTransform.cpp:
(WebCore::RTCRtpSFrameTransform::initializeTransformer):
* Modules/mediastream/RTCRtpSFrameTransform.h:
(WebCore::RTCRtpSFrameTransform::isAttached const):
* Modules/mediastream/RTCRtpSFrameTransform.idl:
* Modules/mediastream/RTCRtpScriptTransform.cpp: Added.
(WebCore::RTCRtpScriptTransform::create):
(WebCore::RTCRtpScriptTransform::RTCRtpScriptTransform):
(WebCore::RTCRtpScriptTransform::~RTCRtpScriptTransform):
(WebCore::RTCRtpScriptTransform::setTransformer):
(WebCore::RTCRtpScriptTransform::initializeBackendForReceiver):
(WebCore::RTCRtpScriptTransform::initializeBackendForSender):
(WebCore::RTCRtpScriptTransform::willClearBackend):
(WebCore::RTCRtpScriptTransform::initializeTransformer):
* Modules/mediastream/RTCRtpScriptTransform.h: Added.
* Modules/mediastream/RTCRtpScriptTransform.idl: Added.
* Modules/mediastream/RTCRtpScriptTransformProvider.idl: Added.
* Modules/mediastream/RTCRtpScriptTransformer.cpp: Added.
(WebCore::RTCRtpReadableStreamSource::create):
(WebCore::RTCRtpReadableStreamSource::close):
(WebCore::RTCRtpReadableStreamSource::enqueue):
(WebCore::RTCRtpWritableStreamSink::create):
(WebCore::RTCRtpWritableStreamSink::RTCRtpWritableStreamSink):
(WebCore::RTCRtpWritableStreamSink::write):
(WebCore::RTCRtpScriptTransformer::create):
(WebCore::RTCRtpScriptTransformer::RTCRtpScriptTransformer):
(WebCore::RTCRtpScriptTransformer::~RTCRtpScriptTransformer):
(WebCore::RTCRtpScriptTransformer::start):
(WebCore::RTCRtpScriptTransformer::clear):
* Modules/mediastream/RTCRtpScriptTransformer.h: Added.
(WebCore::RTCRtpScriptTransformer::setCallback):
(WebCore::RTCRtpScriptTransformer::port):
(WebCore::RTCRtpScriptTransformer::startPendingActivity):
(WebCore::RTCRtpScriptTransformer::activeDOMObjectName const):
(WebCore::RTCRtpScriptTransformer::stopPendingActivity):
* Modules/mediastream/RTCRtpScriptTransformer.idl: Added.
* Modules/mediastream/RTCRtpScriptTransformerConstructor.h: Added.
* Modules/mediastream/RTCRtpScriptTransformerConstructor.idl: Added.
* Modules/mediastream/RTCRtpSender+Transform.idl:
* Modules/mediastream/RTCRtpSender.cpp:
(WebCore::RTCRtpSender::setTransform):
(WebCore::RTCRtpSender::transform):
* Modules/mediastream/RTCRtpSender.h:
* Modules/mediastream/RTCRtpSenderWithTransform.h:
(WebCore::RTCRtpSenderWithTransform::transform):
(WebCore::RTCRtpSenderWithTransform::setTransform):
* Modules/mediastream/RTCRtpTransform.cpp:
(WebCore::RTCRtpTransform::from):
(WebCore::RTCRtpTransform::RTCRtpTransform):
(WebCore::RTCRtpTransform::~RTCRtpTransform):
(WebCore::RTCRtpTransform::isAttached const):
(WebCore::RTCRtpTransform::attachToReceiver):
(WebCore::RTCRtpTransform::attachToSender):
(WebCore::RTCRtpTransform::clearBackend):
(WebCore::RTCRtpTransform::detachFromReceiver):
(WebCore::RTCRtpTransform::detachFromSender):
* Modules/mediastream/RTCRtpTransform.h:
(WebCore::RTCRtpTransform::internalTransform):
* Modules/mediastream/RTCRtpTransformBackend.h:
* Modules/mediastream/RTCRtpTransformableFrame.h:
* Modules/mediastream/libwebrtc/LibWebRTCRtpTransformBackend.cpp:
(WebCore::LibWebRTCRtpTransformBackend::setInputCallback):
(WebCore::LibWebRTCRtpTransformBackend::Transform):
* Modules/streams/WritableStreamSink.h: Added.
* Modules/streams/WritableStreamSink.idl: Added.
* Sources.txt:
* WebCore.xcodeproj/project.pbxproj:
* bindings/js/ReadableStreamDefaultController.cpp:
(WebCore::ReadableStreamDefaultController::enqueue):
* bindings/js/ReadableStreamDefaultController.h:
* bindings/js/WebCoreBuiltinNames.h:
* bindings/js/WritableStream.cpp: Added.
(WebCore::WritableStream::create):
* bindings/js/WritableStream.h: Added.
(WebCore::JSWritableStreamWrapperConverter::toWrapped):
(WebCore::WritableStream::WritableStream):
(WebCore::toJS):
(WebCore::toJSNewlyCreated):
* dom/EventTargetFactory.in:
* testing/Internals.cpp:
* testing/Internals.h:
* testing/Internals.idl:
* testing/MockRTCRtpTransform.cpp:
(WebCore::MockRTCRtpTransformer::transform):
* workers/DedicatedWorkerGlobalScope.cpp:
(WebCore::DedicatedWorkerGlobalScope::registerRTCRtpScriptTransformer):
(WebCore::DedicatedWorkerGlobalScope::createRTCRtpScriptTransformer):
* workers/DedicatedWorkerGlobalScope.h:
* workers/DedicatedWorkerGlobalScope.idl:
* workers/Worker.cpp:
(WebCore::Worker::addRTCRtpScriptTransformer):
(WebCore::Worker::createRTCRtpScriptTransformer):
(WebCore::Worker::postTaskToWorkerGlobalScope):
* workers/Worker.h:
* workers/WorkerGlobalScopeProxy.h:
* workers/WorkerMessagingProxy.cpp:
(WebCore::WorkerMessagingProxy::postTaskToWorkerObject):
(WebCore::WorkerMessagingProxy::postMessageToWorkerGlobalScope):
(WebCore::WorkerMessagingProxy::postTaskToWorkerGlobalScope):
* workers/WorkerMessagingProxy.h:
* workers/WorkerObjectProxy.h:
(WebCore::WorkerObjectProxy::postTaskToWorkerObject):
LayoutTests:
* http/wpt/webrtc/routines.js: Added.
(createConnections):
* http/wpt/webrtc/script-transform.js: Added.
(MockRTCRtpTransformer):
(MockRTCRtpTransformer.prototype.start):
(MockRTCRtpTransformer.prototype.process):
* http/wpt/webrtc/sframe-transform-expected.txt: Added.
* http/wpt/webrtc/sframe-transform.html: Added.
* http/wpt/webrtc/webrtc-transform-expected.txt: Renamed from LayoutTests/webrtc/webrtc-transform-expected.txt.
* http/wpt/webrtc/webrtc-transform.html: Renamed from LayoutTests/webrtc/webrtc-transform.html.
* platform/glib/TestExpectations:
* webrtc/script-transform.js: Added.
(MockRTCRtpTransformer):
(MockRTCRtpTransformer.prototype.start):
Canonical link: https://commits.webkit.org/231818@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@270107 268f45cc-cd09-0410-ab3c-d52691b4dbfc1 parent 514a713 commit 9877c92
71 files changed
Lines changed: 2265 additions & 341 deletions
File tree
- LayoutTests
- http/wpt/webrtc
- platform/glib
- webrtc
- Source/WebCore
- Modules
- mediastream
- libwebrtc
- streams
- WebCore.xcodeproj
- bindings/js
- dom
- testing
- workers
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
1 | 23 | | |
2 | 24 | | |
3 | 25 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
0 commit comments