forked from taskflow/taskflow
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathProfiler.html
More file actions
136 lines (134 loc) · 12.2 KB
/
Copy pathProfiler.html
File metadata and controls
136 lines (134 loc) · 12.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<title>Cookbook » Profile Taskflow Programs | Taskflow QuickStart</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,400i,600,600i%7CSource+Code+Pro:400,400i,600" />
<link rel="stylesheet" href="m-dark+documentation.compiled.css" />
<link rel="icon" href="favicon.ico" type="image/vnd.microsoft.icon" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="theme-color" content="#22272e" />
</head>
<body>
<header><nav id="navigation">
<div class="m-container">
<div class="m-row">
<span id="m-navbar-brand" class="m-col-t-8 m-col-m-none m-left-m">
<a href="https://taskflow.github.io"><img src="taskflow_logo.png" alt="" />Taskflow</a> <span class="m-breadcrumb">|</span> <a href="index.html" class="m-thin">QuickStart</a>
</span>
<div class="m-col-t-4 m-hide-m m-text-right m-nopadr">
<a href="#search" class="m-doc-search-icon" title="Search" onclick="return showSearch()"><svg style="height: 0.9rem;" viewBox="0 0 16 16">
<path id="m-doc-search-icon-path" d="m6 0c-3.31 0-6 2.69-6 6 0 3.31 2.69 6 6 6 1.49 0 2.85-0.541 3.89-1.44-0.0164 0.338 0.147 0.759 0.5 1.15l3.22 3.79c0.552 0.614 1.45 0.665 2 0.115 0.55-0.55 0.499-1.45-0.115-2l-3.79-3.22c-0.392-0.353-0.812-0.515-1.15-0.5 0.895-1.05 1.44-2.41 1.44-3.89 0-3.31-2.69-6-6-6zm0 1.56a4.44 4.44 0 0 1 4.44 4.44 4.44 4.44 0 0 1-4.44 4.44 4.44 4.44 0 0 1-4.44-4.44 4.44 4.44 0 0 1 4.44-4.44z"/>
</svg></a>
<a id="m-navbar-show" href="#navigation" title="Show navigation"></a>
<a id="m-navbar-hide" href="#" title="Hide navigation"></a>
</div>
<div id="m-navbar-collapse" class="m-col-t-12 m-show-m m-col-m-none m-right-m">
<div class="m-row">
<ol class="m-col-t-6 m-col-m-none">
<li><a href="pages.html">Handbook</a></li>
<li><a href="namespaces.html">Namespaces</a></li>
</ol>
<ol class="m-col-t-6 m-col-m-none" start="3">
<li><a href="annotated.html">Classes</a></li>
<li><a href="files.html">Files</a></li>
<li class="m-show-m"><a href="#search" class="m-doc-search-icon" title="Search" onclick="return showSearch()"><svg style="height: 0.9rem;" viewBox="0 0 16 16">
<use href="#m-doc-search-icon-path" />
</svg></a></li>
</ol>
</div>
</div>
</div>
</div>
</nav></header>
<main><article>
<div class="m-container m-container-inflatable">
<div class="m-row">
<div class="m-col-l-10 m-push-l-1">
<h1>
<span class="m-breadcrumb"><a href="Cookbook.html">Cookbook</a> »</span>
Profile Taskflow Programs
</h1>
<nav class="m-block m-default">
<h3>Contents</h3>
<ul>
<li><a href="#ProfilerEnableTFProf">Enable Taskflow Profiler</a></li>
<li><a href="#ProfilerEnableTFProfServer">Enable Taskflow Profiler on a HTTP Server</a></li>
<li><a href="#ProfilerDisplayProfileSummary">Display Profile Summary</a></li>
</ul>
</nav>
<p>Taskflow comes with a built-in profiler, <em>TFProf</em>, for you to profile and visualize taskflow programs.</p><img class="m-image" src="tfprof.png" alt="Image" style="width: 100%;" /><section id="ProfilerEnableTFProf"><h2><a href="#ProfilerEnableTFProf">Enable Taskflow Profiler</a></h2><p>All taskflow programs come with a lightweight profiling module to observer worker activities in every executor. To enable the profiler, set the environment variable <code>TF_ENABLE_PROFILER</code> to a file name in which the profiling result will be stored.</p><pre class="m-console"><span class="go">~$ TF_ENABLE_PROFILER=result.json ./my_taskflow</span>
<span class="go">~$ cat result.json</span>
<span class="go">[</span>
<span class="go">{"executor":"0","data":[{"worker":12,"level":0,"data":[{"span":[72,117],"name":"12_0","type":"static"},{"span":[121,123],"name":"12_1","type":"static"},{"span":[123,125],"name":"12_2","type":"static"},{"span":[125,127],"name":"12_3","type":"static"}]}]}</span>
<span class="go">]</span></pre><p>When the program finishes, it generates and saves the profiling data to <code>result.json</code> in JavaScript Object Notation (JSON) format. You can then paste the JSON data to our web-based interface, <a href="https://taskflow.github.io/tfprof/">Taskflow Profiler</a>, to visualize the execution timelines of tasks and workers. The web interface supports the following features:</p><ul><li>zoom into a selected window</li><li>double click to zoom back to the previously selected window</li><li>filter workers</li><li>mouse over to show the tooltip of the task</li><li>rank tasks in decreasing order of criticality (i.e., execution time)</li></ul><p>TFProf implements a clustering-based algorithm to efficiently visualize tasks and their execution timelines in a browser. Without losing much visual accuracy, each <em>clustered</em> task indicates a group of adjacent tasks clustered by the algorithm, and you can zoom in to see these tasks.</p></section><section id="ProfilerEnableTFProfServer"><h2><a href="#ProfilerEnableTFProfServer">Enable Taskflow Profiler on a HTTP Server</a></h2><p>When profiling large taskflow programs, the method in the previous section may not work because of the limitation of processing large JSON files. For example, a taskflow program of a million tasks can produce several GBs of profiling data, and the profile may respond to your requests very slowly. To solve this problem, we have implemented a C++-based http server optimized for our profiling data. To compile the server, enable the cmake option <code>TF_BUILD_PROFILER</code>. You may visit <a href="install.html" class="m-doc">Building and Installing</a> to understand Taskflow's build environment.</p><pre class="m-console"><span class="gp"># </span>under the build directory
<span class="go">~$ cmake ../ -DTF_BUILD_PROFILER=ON</span>
<span class="go">~$ make</span></pre><p>After successfully compiling the server, you can find the executable at <code>tfprof/server/tfprof</code>. Now, generate profiling data from running a taskflow program but specify the output file with extension .tfp.</p><pre class="m-console"><span class="go">~$ TF_ENABLE_PROFILER=my_taskflow.tfp ./my_taskflow</span>
<span class="go">~$ ls</span>
<span class="go">my_taskflow.tfp # my_taskflow.tfp is of binary format</span></pre><p>Launch the server program <code>tfprof/server/tfprof</code> and pass (1) the directory of <code>index.html</code> (default at <code>tfprof/</code>) via the option <code>–mount</code> and (2) the <code>my_taskflow.tfp</code> via the option <code>–input</code>.</p><pre class="m-console"><span class="gp"># </span>under the build/ directory
<span class="go">~$ ./tfprof/server/tfprof --mount ../tfprof/ --input my_taskflow.tfp</span></pre><p>Now, open your favorite browser at <code>localhost:8080</code> to visualize and profile your <code>my_taskflow</code> program.</p><img class="m-image" src="tfprof-local.png" alt="Image" /><p>The compiled profiler is a more powerful version than the pure JavaScript-based interface and it is able to more efficiently handle large profiling data under different queries. We currently support the following two view types:</p><ul><li>Cluster: visualize the profiling data using a clustering algorithm with a limit</li><li>Criticality: visualize the top-limit tasks in decreasing order of their execution times</li></ul></section><section id="ProfilerDisplayProfileSummary"><h2><a href="#ProfilerDisplayProfileSummary">Display Profile Summary</a></h2><p>You can display a profile summary by specifying only the environment variable <code>TF_ENABLE_PROFILER</code> without any value. The Taskflow will generate a separate summary report of tasks and workers for each executor created by the program.</p><pre class="m-console"><span class="gp"># </span><span class="nb">enable</span> the environment variable without any value
<span class="go">~$ TF_ENABLE_PROFILER= ./my_taskflow_program </span>
<span class="gp"># </span>your program output
<span class="go">...</span>
<span class="go">... </span>
<span class="go">...</span>
<span class="gp"># </span>Taskflow profile summary
<span class="go">==Observer 0: 1 workers completed 18 tasks in 28 us</span>
<span class="go"> -Task- Count Time (us) Avg (us) Min (us) Max (us)</span>
<span class="go"> static 7 5 0.714286 0 4</span>
<span class="go"> condition 11 0 0.000000 0 0</span>
<span class="go"> -Worker- Level Task Count Time (us) Avg (us) Min (us) Max (us)</span>
<span class="go"> 14 0 static 7 5 0.714286 0 4</span>
<span class="go"> condition 11 0 0.000000 0 0</span>
<span class="go"> 18 5 0.277778 0 4</span></pre><p>The report consists of two sections, task summary and worker summary. In the first section, the summary reports for each task type the number of executions (<code>Count</code>), the total execution time (<code>Time</code>), average execution time per task (<code>Avg</code>), and the minimum (<code>Min</code>) and the maximum (<code>Max</code>) execution time among all tasks. Similarly in the second section, the summary reports for each worker the task execution statistics.</p></section>
</div>
</div>
</div>
</article></main>
<div class="m-doc-search" id="search">
<a href="#!" onclick="return hideSearch()"></a>
<div class="m-container">
<div class="m-row">
<div class="m-col-m-8 m-push-m-2">
<div class="m-doc-search-header m-text m-small">
<div><span class="m-label m-default">Tab</span> / <span class="m-label m-default">T</span> to search, <span class="m-label m-default">Esc</span> to close</div>
<div id="search-symbolcount">…</div>
</div>
<div class="m-doc-search-content">
<form>
<input type="search" name="q" id="search-input" placeholder="Loading …" disabled="disabled" autofocus="autofocus" autocomplete="off" spellcheck="false" />
</form>
<noscript class="m-text m-danger m-text-center">Unlike everything else in the docs, the search functionality <em>requires</em> JavaScript.</noscript>
<div id="search-help" class="m-text m-dim m-text-center">
<p class="m-noindent">Search for symbols, directories, files, pages or
modules. You can omit any prefix from the symbol or file path; adding a
<code>:</code> or <code>/</code> suffix lists all members of given symbol or
directory.</p>
<p class="m-noindent">Use <span class="m-label m-dim">↓</span>
/ <span class="m-label m-dim">↑</span> to navigate through the list,
<span class="m-label m-dim">Enter</span> to go.
<span class="m-label m-dim">Tab</span> autocompletes common prefix, you can
copy a link to the result using <span class="m-label m-dim">⌘</span>
<span class="m-label m-dim">L</span> while <span class="m-label m-dim">⌘</span>
<span class="m-label m-dim">M</span> produces a Markdown link.</p>
</div>
<div id="search-notfound" class="m-text m-warning m-text-center">Sorry, nothing was found.</div>
<ul id="search-results"></ul>
</div>
</div>
</div>
</div>
</div>
<script src="search-v2.js"></script>
<script src="searchdata-v2.js" async="async"></script>
<footer><nav>
<div class="m-container">
<div class="m-row">
<div class="m-col-l-10 m-push-l-1">
<p>Taskflow handbook is part of the <a href="https://taskflow.github.io">Taskflow project</a>, copyright © <a href="https://tsung-wei-huang.github.io/">Dr. Tsung-Wei Huang</a>, 2018–2023.<br />Generated by <a href="https://doxygen.org/">Doxygen</a> 1.8.14 and <a href="https://mcss.mosra.cz/">m.css</a>.</p>
</div>
</div>
</div>
</nav></footer>
</body>
</html>