Skip to content

Commit d86206d

Browse files
cdumezcdumez
committed
AnalyserNode's getByteFrequencyData() / getFloatFrequencyData() should only do FFT analysis once per render quantum
https://bugs.webkit.org/show_bug.cgi?id=216735 Reviewed by Darin Adler. Source/WebCore: AnalyserNode's getByteFrequencyData() / getFloatFrequencyData() should only do FFT analysis once per render quantum: - https://www.w3.org/TR/webaudio/#dom-analysernode-getbytefrequencydata """ If another call to getByteFrequencyData() or getFloatFrequencyData() occurs within the same render quantum as a previous call, the current frequency data is not updated with the same data. Instead, the previously computed data is returned. """ No new tests, rebaselined existing test. * Modules/webaudio/RealtimeAnalyser.cpp: (WebCore::RealtimeAnalyser::writeInput): (WebCore::RealtimeAnalyser::doFFTAnalysisIfNecessary): (WebCore::RealtimeAnalyser::getFloatFrequencyData): (WebCore::RealtimeAnalyser::getByteFrequencyData): * Modules/webaudio/RealtimeAnalyser.h: LayoutTests: Rebaseline webaudio tests now that more checks are passing. * webaudio/Analyser/realtimeanalyser-freq-data-smoothing-expected.txt: * webaudio/Analyser/realtimeanalyser-multiple-calls-expected.txt: Canonical link: https://commits.webkit.org/229568@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@267349 268f45cc-cd09-0410-ab3c-d52691b4dbfc
1 parent adc85d7 commit d86206d

6 files changed

Lines changed: 85 additions & 61 deletions

File tree

‎LayoutTests/ChangeLog‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,15 @@
1+
2020-09-21 Chris Dumez <cdumez@apple.com>
2+
3+
AnalyserNode's getByteFrequencyData() / getFloatFrequencyData() should only do FFT analysis once per render quantum
4+
https://bugs.webkit.org/show_bug.cgi?id=216735
5+
6+
Reviewed by Darin Adler.
7+
8+
Rebaseline webaudio tests now that more checks are passing.
9+
10+
* webaudio/Analyser/realtimeanalyser-freq-data-smoothing-expected.txt:
11+
* webaudio/Analyser/realtimeanalyser-multiple-calls-expected.txt:
12+
113
2020-09-21 Chris Dumez <cdumez@apple.com>
214

315
AnalyserNode should downmix input audio to mono

‎LayoutTests/webaudio/Analyser/realtimeanalyser-freq-data-smoothing-expected.txt‎

Lines changed: 26 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -17,39 +17,39 @@ FAIL X First 512-point FFT at frame 512 does not equal [-100,-100,-96.0621261596
1717
assert_true: expected true got false
1818
FAIL X 512-point byte FFT does not equal [0,0,14,28,43,209,255,255,252,186,37,53,197,242,250,225...] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0}.
1919
Index Actual Expected AbsError RelError Test threshold
20-
[0] 1.3000000000000000e+1 0.0000000000000000e+0 1.3000000000000000e+1 Infinity 0.0000000000000000e+0
21-
[1] 2.8000000000000000e+1 0.0000000000000000e+0 2.8000000000000000e+1 Infinity 0.0000000000000000e+0
22-
[2] 4.9000000000000000e+1 1.4000000000000000e+1 3.5000000000000000e+1 2.5000000000000000e+0 0.0000000000000000e+0
23-
[3] 6.3000000000000000e+1 2.8000000000000000e+1 3.5000000000000000e+1 1.2500000000000000e+0 0.0000000000000000e+0
24-
[4] 7.7000000000000000e+1 4.3000000000000000e+1 3.4000000000000000e+1 7.9069767441860461e-1 0.0000000000000000e+0
25-
...and 195 more errors.
26-
Max AbsError of 3.5000000000000000e+1 at index of 2.
27-
Max RelError of Infinity at index of 0.
20+
[1] 1.6000000000000000e+1 0.0000000000000000e+0 1.6000000000000000e+1 Infinity 0.0000000000000000e+0
21+
[2] 3.6000000000000000e+1 1.4000000000000000e+1 2.2000000000000000e+1 1.5714285714285714e+0 0.0000000000000000e+0
22+
[3] 5.0000000000000000e+1 2.8000000000000000e+1 2.2000000000000000e+1 7.8571428571428570e-1 0.0000000000000000e+0
23+
[4] 6.5000000000000000e+1 4.3000000000000000e+1 2.2000000000000000e+1 5.1162790697674421e-1 0.0000000000000000e+0
24+
[5] 2.3100000000000000e+2 2.0900000000000000e+2 2.2000000000000000e+1 1.0526315789473684e-1 0.0000000000000000e+0
25+
...and 189 more errors.
26+
Max AbsError of 2.2000000000000000e+1 at index of 2.
27+
Max RelError of Infinity at index of 1.
2828
assert_true: expected true got false
2929
FAIL X Smoothed 512-point FFT at frame 1536 does not equal [-94.55278778076172,-93.13629150390625,-90.07610321044922,-87.21340942382812,-85.55162048339844,-38.930355072021484,-24.825944900512695,-21.471542358398438,-27.15943145751953,-45.10898971557617,-87.53993225097656,-78.7408447265625,-42.376895904541016,-29.92169761657715,-27.697038650512695,-34.60657501220703...] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0.000025332}.
3030
Index Actual Expected AbsError RelError Test threshold
31-
[0] -8.7954093933105469e+1 -9.4552787780761719e+1 6.5986938476562500e+0 6.9788464227586319e-2 2.3952112200622561e-3
32-
[1] -8.6332389831542969e+1 -9.3136291503906250e+1 6.8039016723632813e+0 7.3053173607174574e-2 2.3593285363769532e-3
33-
[2] -8.3028343200683594e+1 -9.0076103210449219e+1 7.0477600097656250e+0 7.8242283564372198e-2 2.2818078465270996e-3
34-
[3] -8.0046691894531250e+1 -8.7213409423828125e+1 7.1667175292968750e+0 8.2174491017419291e-2 2.2092900875244140e-3
35-
[4] -7.8052825927734375e+1 -8.5551620483398438e+1 7.4987945556640625e+0 8.7652279562831045e-2 2.1671936500854494e-3
36-
...and 201 more errors.
37-
Max AbsError of 7.6593704223632813e+0 at index of 79.
38-
[79] -9.0711067199707031e+1 -9.8370437622070313e+1 7.6593704223632813e+0 7.7862522598403400e-2 2.4919199258422852e-3
39-
Max RelError of 3.4275040169589932e-1 at index of 7.
40-
[7] -1.4112162590026855e+1 -2.1471542358398438e+1 7.3593797683715820e+0 3.4275040169589932e-1 5.4391711102294929e-4
31+
[0] -8.8532302856445313e+1 -9.4552787780761719e+1 6.0204849243164063e+0 6.3673267236456568e-2 2.3952112200622561e-3
32+
[1] -8.7115928649902344e+1 -9.3136291503906250e+1 6.0203628540039063e+0 6.4640354010137985e-2 2.3593285363769532e-3
33+
[2] -8.4055267333984375e+1 -9.0076103210449219e+1 6.0208358764648438e+0 6.6841655687503151e-2 2.2818078465270996e-3
34+
[3] -8.1192565917968750e+1 -8.7213409423828125e+1 6.0208435058593750e+0 6.9035754313881717e-2 2.2092900875244140e-3
35+
[4] -7.9531051635742188e+1 -8.5551620483398438e+1 6.0205688476562500e+0 7.0373522016740298e-2 2.1671936500854494e-3
36+
...and 199 more errors.
37+
Max AbsError of 6.0210571289062500e+0 at index of 183.
38+
[183] -9.0905830383300781e+1 -9.6926887512207031e+1 6.0210571289062500e+0 6.2119575728127596e-2 2.4553519144592285e-3
39+
Max RelError of 2.8039901674224449e-1 at index of 7.
40+
[7] -1.5450942993164063e+1 -2.1471542358398438e+1 6.0205993652343750e+0 2.8039901674224449e-1 5.4391711102294929e-4
4141
assert_true: expected true got false
4242
FAIL X 512-point byte FFT does not equal [19,25,36,46,52,222,255,255,255,199,45,77,209,255,255,238...] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0}.
4343
Index Actual Expected AbsError RelError Test threshold
44-
[0] 5.2000000000000000e+1 1.9000000000000000e+1 3.3000000000000000e+1 1.7368421052631580e+0 0.0000000000000000e+0
45-
[1] 5.6000000000000000e+1 2.5000000000000000e+1 3.1000000000000000e+1 1.2400000000000000e+0 0.0000000000000000e+0
46-
[2] 6.6000000000000000e+1 3.6000000000000000e+1 3.0000000000000000e+1 8.3333333333333337e-1 0.0000000000000000e+0
47-
[3] 7.6000000000000000e+1 4.6000000000000000e+1 3.0000000000000000e+1 6.5217391304347827e-1 0.0000000000000000e+0
48-
[4] 8.0000000000000000e+1 5.2000000000000000e+1 2.8000000000000000e+1 5.3846153846153844e-1 0.0000000000000000e+0
49-
...and 196 more errors.
50-
Max AbsError of 3.3000000000000000e+1 at index of 0.
44+
[0] 4.1000000000000000e+1 1.9000000000000000e+1 2.2000000000000000e+1 1.1578947368421053e+0 0.0000000000000000e+0
45+
[1] 4.6000000000000000e+1 2.5000000000000000e+1 2.1000000000000000e+1 8.3999999999999997e-1 0.0000000000000000e+0
46+
[2] 5.8000000000000000e+1 3.6000000000000000e+1 2.2000000000000000e+1 6.1111111111111116e-1 0.0000000000000000e+0
47+
[3] 6.8000000000000000e+1 4.6000000000000000e+1 2.2000000000000000e+1 4.7826086956521741e-1 0.0000000000000000e+0
48+
[4] 7.4000000000000000e+1 5.2000000000000000e+1 2.2000000000000000e+1 4.2307692307692307e-1 0.0000000000000000e+0
49+
...and 193 more errors.
50+
Max AbsError of 2.2000000000000000e+1 at index of 0.
5151
Max RelError of Infinity at index of 51.
52-
[51] 2.2000000000000000e+1 0.0000000000000000e+0 2.2000000000000000e+1 Infinity 0.0000000000000000e+0
52+
[51] 1.5000000000000000e+1 0.0000000000000000e+0 1.5000000000000000e+1 Infinity 0.0000000000000000e+0
5353
assert_true: expected true got false
5454
FAIL < [smoothing test] 4 out of 4 assertions were failed. assert_true: expected true got false
5555
FAIL # AUDIT TASK RUNNER FINISHED: 1 out of 1 tasks were failed. assert_true: expected true got false

‎LayoutTests/webaudio/Analyser/realtimeanalyser-multiple-calls-expected.txt‎

Lines changed: 6 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -3,34 +3,10 @@ PASS # AUDIT TASK RUNNER STARTED.
33
PASS Executing "test"
44
PASS Audit report
55
PASS > [test]
6-
FAIL X Second call to getFloatFrequencyData expected to be equal to the array [-52.5184440612793,-35.278839111328125,-27.40174102783203,-29.20952796936035,-35.99977493286133,-33.581512451171875,-40.76682662963867,-36.675636291503906,-41.62258529663086,-40.120052337646484,-41.18754959106445,-44.266780853271484,-41.43907165527344,-47.687599182128906,-42.71406173706055,-47.281829833984375...] but differs in 128 places:
7-
Index Actual Expected
8-
[0] -4.7412990570068359e+1 -5.2518444061279297e+1
9-
[1] -3.0173389434814453e+1 -3.5278839111328125e+1
10-
[2] -2.2296291351318359e+1 -2.7401741027832031e+1
11-
[3] -2.4104078292846680e+1 -2.9209527969360352e+1
12-
...and 124 more errors. assert_true: expected true got false
13-
FAIL X Second call to getByteFrequencyData expected to be equal to the array [199,255,255,255,255,255,252,255,246,248,244,236,242,226,237,225...] but differs in 114 places:
14-
Index Actual Expected
15-
[0] 2.0500000000000000e+2 1.9900000000000000e+2
16-
[6] 2.5500000000000000e+2 2.5200000000000000e+2
17-
[8] 2.5400000000000000e+2 2.4600000000000000e+2
18-
[9] 2.5500000000000000e+2 2.4800000000000000e+2
19-
...and 110 more errors. assert_true: expected true got false
20-
FAIL X Output of getByteFrequencyData after getFloatFrequencyData expected to be equal to the array [195,255,255,255,255,255,255,255,255,255,255,255,255,248,252,245...] but differs in 110 places:
21-
Index Actual Expected
22-
[0] 2.0000000000000000e+2 1.9500000000000000e+2
23-
[11] 2.5000000000000000e+2 2.5500000000000000e+2
24-
[12] 2.5300000000000000e+2 2.5500000000000000e+2
25-
[13] 2.4300000000000000e+2 2.4800000000000000e+2
26-
...and 106 more errors. assert_true: expected true got false
27-
FAIL X Output of getFloatFrequenycData (converted to byte) after getByteFrequencyData expected to be equal to the array [209,255,255,255,255,255,255,255,255,255,255,255,255,251,255,248...] but differs in 108 places:
28-
Index Actual Expected
29-
[0] 2.0300000000000000e+2 2.0900000000000000e+2
30-
[13] 2.5000000000000000e+2 2.5100000000000000e+2
31-
[14] 2.5400000000000000e+2 2.5500000000000000e+2
32-
[15] 2.4700000000000000e+2 2.4800000000000000e+2
33-
...and 104 more errors. assert_true: expected true got false
34-
FAIL < [test] 4 out of 4 assertions were failed. assert_true: expected true got false
35-
FAIL # AUDIT TASK RUNNER FINISHED: 1 out of 1 tasks were failed. assert_true: expected true got false
6+
PASS Second call to getFloatFrequencyData is identical to the array [expected array].
7+
PASS Second call to getByteFrequencyData is identical to the array [189,254,255,255,255,255,245,250,238,239,236,228,233,219,228,218...].
8+
PASS Output of getByteFrequencyData after getFloatFrequencyData is identical to the array [185,255,255,255,255,255,255,255,251,250,247,241,243,234,238,231...].
9+
PASS Output of getFloatFrequenycData (converted to byte) after getByteFrequencyData is identical to the array [198,255,255,255,255,255,255,255,255,255,253,247,249,239,244,237...].
10+
PASS < [test] All assertions passed. (total 4 assertions)
11+
PASS # AUDIT TASK RUNNER FINISHED: 1 tasks ran successfully.
3612

‎Source/WebCore/ChangeLog‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,28 @@
1+
2020-09-21 Chris Dumez <cdumez@apple.com>
2+
3+
AnalyserNode's getByteFrequencyData() / getFloatFrequencyData() should only do FFT analysis once per render quantum
4+
https://bugs.webkit.org/show_bug.cgi?id=216735
5+
6+
Reviewed by Darin Adler.
7+
8+
AnalyserNode's getByteFrequencyData() / getFloatFrequencyData() should only do FFT analysis once per render quantum:
9+
- https://www.w3.org/TR/webaudio/#dom-analysernode-getbytefrequencydata
10+
11+
"""
12+
If another call to getByteFrequencyData() or getFloatFrequencyData() occurs within the same render quantum as a
13+
previous call, the current frequency data is not updated with the same data. Instead, the previously computed data
14+
is returned.
15+
"""
16+
17+
No new tests, rebaselined existing test.
18+
19+
* Modules/webaudio/RealtimeAnalyser.cpp:
20+
(WebCore::RealtimeAnalyser::writeInput):
21+
(WebCore::RealtimeAnalyser::doFFTAnalysisIfNecessary):
22+
(WebCore::RealtimeAnalyser::getFloatFrequencyData):
23+
(WebCore::RealtimeAnalyser::getByteFrequencyData):
24+
* Modules/webaudio/RealtimeAnalyser.h:
25+
126
2020-09-21 Antoine Quint <graouts@webkit.org>
227

328
Remove dedicated HashSet<Element*> for DocumentTimeline::runningAnimationsForElementAreAllAccelerated()

‎Source/WebCore/Modules/webaudio/RealtimeAnalyser.cpp‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,9 @@ void RealtimeAnalyser::writeInput(AudioBus* bus, size_t framesToProcess)
120120
m_writeIndex += framesToProcess;
121121
if (m_writeIndex >= InputBufferSize)
122122
m_writeIndex = 0;
123+
124+
// A new render quantum has been processed so we should do the FFT analysis again.
125+
m_shouldDoFFTAnalysis = true;
123126
}
124127

125128
namespace {
@@ -143,10 +146,15 @@ void applyWindow(float* p, size_t n)
143146

144147
} // namespace
145148

146-
void RealtimeAnalyser::doFFTAnalysis()
149+
void RealtimeAnalyser::doFFTAnalysisIfNecessary()
147150
{
148151
ASSERT(isMainThread());
149152

153+
if (!m_shouldDoFFTAnalysis)
154+
return;
155+
156+
m_shouldDoFFTAnalysis = false;
157+
150158
// Unroll the input buffer into a temporary buffer, where we'll apply an analysis window followed by an FFT.
151159
size_t fftSize = this->fftSize();
152160

@@ -197,7 +205,7 @@ void RealtimeAnalyser::getFloatFrequencyData(Float32Array& destinationArray)
197205
{
198206
ASSERT(isMainThread());
199207

200-
doFFTAnalysis();
208+
doFFTAnalysisIfNecessary();
201209

202210
// Convert from linear magnitude to floating-point decibels.
203211
unsigned sourceLength = magnitudeBuffer().size();
@@ -215,7 +223,7 @@ void RealtimeAnalyser::getByteFrequencyData(Uint8Array& destinationArray)
215223
{
216224
ASSERT(isMainThread());
217225

218-
doFFTAnalysis();
226+
doFFTAnalysisIfNecessary();
219227

220228
// Convert from linear magnitude to unsigned-byte decibels.
221229
unsigned sourceLength = magnitudeBuffer().size();

‎Source/WebCore/Modules/webaudio/RealtimeAnalyser.h‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -86,9 +86,9 @@ class RealtimeAnalyser {
8686

8787
size_t m_fftSize;
8888
std::unique_ptr<FFTFrame> m_analysisFrame;
89-
void doFFTAnalysis();
89+
void doFFTAnalysisIfNecessary();
9090

91-
// doFFTAnalysis() stores the floating-point magnitude analysis data here.
91+
// doFFTAnalysisIfNecessary() stores the floating-point magnitude analysis data here.
9292
AudioFloatArray m_magnitudeBuffer;
9393
AudioFloatArray& magnitudeBuffer() { return m_magnitudeBuffer; }
9494

@@ -98,6 +98,9 @@ class RealtimeAnalyser {
9898
// The range used when converting when using getByteFrequencyData().
9999
double m_minDecibels;
100100
double m_maxDecibels;
101+
102+
// We should only do the FFT analysis once per render quantum.
103+
bool m_shouldDoFFTAnalysis { true };
101104
};
102105

103106
} // namespace WebCore

0 commit comments

Comments
 (0)