Skip to content

Commit 2f13072

Browse files
jkbradleyjkbradley
committed
Copied stuff from Spark docs and build to generate doc pages and API docs. Made minimal edits to make it reasonable, but need to edit text eventually. Also fixed merge conflict.
1 parent 991a5a4 commit 2f13072

41 files changed

Lines changed: 10673 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,9 @@
33
*.pyc
44
build/*.jar
55

6+
docs/_site
7+
docs/api
8+
69
# sbt specific
710
.cache/
811
.history/

‎docs/README.md‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
Welcome to the DFGraph Spark Package documentation!
2+
3+
This readme will walk you through navigating and building the DFGraph documentation, which is
4+
included here with the source code.
5+
6+
Read on to learn more about viewing documentation in plain text (i.e., markdown) or building the
7+
documentation yourself. Why build it yourself? So that you have the docs that correspond to
8+
whichever version of DFGraph you currently have checked out of revision control.
9+
10+
## Generating the Documentation HTML
11+
12+
We include the DFGraph documentation as part of the source (as opposed to using a hosted wiki, such as
13+
the github wiki, as the definitive documentation) to enable the documentation to evolve along with
14+
the source code and be captured by revision control (currently git). This way the code automatically
15+
includes the version of the documentation that is relevant regardless of which version or release
16+
you have checked out or downloaded.
17+
18+
In this directory you will find textfiles formatted using Markdown, with an ".md" suffix. You can
19+
read those text files directly if you want. Start with index.md.
20+
21+
The markdown code can be compiled to HTML using the [Jekyll tool](http://jekyllrb.com).
22+
`Jekyll` and a few dependencies must be installed for this to work. We recommend
23+
installing via the Ruby Gem dependency manager. Since the exact HTML output
24+
varies between versions of Jekyll and its dependencies, we list specific versions here
25+
in some cases:
26+
27+
$ sudo gem install jekyll
28+
$ sudo gem install jekyll-redirect-from
29+
30+
Execute `jekyll build` from the `docs/` directory to compile the site. Compiling the site with Jekyll will create a directory
31+
called `_site` containing index.html as well as the rest of the compiled files.
32+
33+
You can modify the default Jekyll build as follows:
34+
35+
# Skip generating API docs (which takes a while)
36+
$ SKIP_API=1 jekyll build
37+
# Serve content locally on port 4000
38+
$ jekyll serve --watch
39+
# Build the site with extra features used on the live page
40+
$ PRODUCTION=1 jekyll build
41+
42+
## Pygments
43+
44+
We also use pygments (http://pygments.org) for syntax highlighting in documentation markdown pages,
45+
so you will also need to install that (it requires Python) by running `sudo pip install Pygments`.
46+
47+
To mark a block of code in your markdown to be syntax highlighted by jekyll during the compile
48+
phase, use the following sytax:
49+
50+
{% highlight scala %}
51+
// Your scala code goes here, you can replace scala with many other
52+
// supported languages too.
53+
{% endhighlight %}
54+
55+
## Sphinx
56+
57+
We use Sphinx to generate Python API docs, so you will need to install it by running
58+
`sudo pip install sphinx`.
59+
60+
## API Docs (Scaladoc, Sphinx)
61+
62+
You can build just the scaladoc by running `build/sbt unidoc` from the DFGRAPH_PROJECT_ROOT directory.
63+
64+
Similarly, you can build just the Python docs by running `make html` from the
65+
DFGRAPH_PROJECT_ROOT/python/docs directory. Documentation is only generated for classes that are listed as
66+
public in `__init__.py`.
67+
68+
When you run `jekyll` in the `docs` directory, it will also copy over the scaladoc for the various
69+
subprojects into the `docs` directory (and then also into the `_site` directory). We use a
70+
jekyll plugin to run `build/sbt unidoc` before building the site so if you haven't run it (recently) it
71+
may take some time as it generates all of the scaladoc. The jekyll plugin also generates the
72+
Python docs [Sphinx](http://sphinx-doc.org/).
73+
74+
NOTE: To skip the step of building and copying over the Scala, Python API docs, run `SKIP_API=1
75+
jekyll`.

‎docs/_config.yml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
highlighter: pygments
2+
markdown: kramdown
3+
gems:
4+
- jekyll-redirect-from
5+
6+
# For some reason kramdown seems to behave differently on different
7+
# OS/packages wrt encoding. So we hard code this config.
8+
kramdown:
9+
entity_output: numeric
10+
11+
include:
12+
- _static
13+
- _modules
14+
15+
# These allow the documentation to be updated with newer releases
16+
# of Spark, Scala, and Mesos.
17+
DFGRAPH_VERSION: 0.1
18+
#SCALA_BINARY_VERSION: "2.10"
19+
#SCALA_VERSION: "2.10.4"
20+
#MESOS_VERSION: 0.21.0
21+
#SPARK_ISSUE_TRACKER_URL: https://issues.apache.org/jira/browse/SPARK
22+
#SPARK_GITHUB_URL: https://github.com/apache/spark

‎docs/_layouts/404.html‎

Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
<!DOCTYPE html>
2+
<html lang="en">
3+
<head>
4+
<meta charset="utf-8">
5+
<title>Page Not Found :(</title>
6+
<style>
7+
::-moz-selection {
8+
background: #b3d4fc;
9+
text-shadow: none;
10+
}
11+
12+
::selection {
13+
background: #b3d4fc;
14+
text-shadow: none;
15+
}
16+
17+
html {
18+
padding: 30px 10px;
19+
font-size: 20px;
20+
line-height: 1.4;
21+
color: #737373;
22+
background: #f0f0f0;
23+
-webkit-text-size-adjust: 100%;
24+
-ms-text-size-adjust: 100%;
25+
}
26+
27+
html,
28+
input {
29+
font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
30+
}
31+
32+
body {
33+
max-width: 500px;
34+
_width: 500px;
35+
padding: 30px 20px 50px;
36+
border: 1px solid #b3b3b3;
37+
border-radius: 4px;
38+
margin: 0 auto;
39+
box-shadow: 0 1px 10px #a7a7a7, inset 0 1px 0 #fff;
40+
background: #fcfcfc;
41+
}
42+
43+
h1 {
44+
margin: 0 10px;
45+
font-size: 50px;
46+
text-align: center;
47+
}
48+
49+
h1 span {
50+
color: #bbb;
51+
}
52+
53+
h3 {
54+
margin: 1.5em 0 0.5em;
55+
}
56+
57+
p {
58+
margin: 1em 0;
59+
}
60+
61+
ul {
62+
padding: 0 0 0 40px;
63+
margin: 1em 0;
64+
}
65+
66+
.container {
67+
max-width: 380px;
68+
_width: 380px;
69+
margin: 0 auto;
70+
}
71+
72+
/* google search */
73+
74+
#goog-fixurl ul {
75+
list-style: none;
76+
padding: 0;
77+
margin: 0;
78+
}
79+
80+
#goog-fixurl form {
81+
margin: 0;
82+
}
83+
84+
#goog-wm-qt,
85+
#goog-wm-sb {
86+
border: 1px solid #bbb;
87+
font-size: 16px;
88+
line-height: normal;
89+
vertical-align: top;
90+
color: #444;
91+
border-radius: 2px;
92+
}
93+
94+
#goog-wm-qt {
95+
width: 220px;
96+
height: 20px;
97+
padding: 5px;
98+
margin: 5px 10px 0 0;
99+
box-shadow: inset 0 1px 1px #ccc;
100+
}
101+
102+
#goog-wm-sb {
103+
display: inline-block;
104+
height: 32px;
105+
padding: 0 10px;
106+
margin: 5px 0 0;
107+
white-space: nowrap;
108+
cursor: pointer;
109+
background-color: #f5f5f5;
110+
background-image: -webkit-linear-gradient(rgba(255,255,255,0), #f1f1f1);
111+
background-image: -moz-linear-gradient(rgba(255,255,255,0), #f1f1f1);
112+
background-image: -ms-linear-gradient(rgba(255,255,255,0), #f1f1f1);
113+
background-image: -o-linear-gradient(rgba(255,255,255,0), #f1f1f1);
114+
-webkit-appearance: none;
115+
-moz-appearance: none;
116+
appearance: none;
117+
*overflow: visible;
118+
*display: inline;
119+
*zoom: 1;
120+
}
121+
122+
#goog-wm-sb:hover,
123+
#goog-wm-sb:focus {
124+
border-color: #aaa;
125+
box-shadow: 0 1px 1px rgba(0, 0, 0, 0.1);
126+
background-color: #f8f8f8;
127+
}
128+
129+
#goog-wm-qt:hover,
130+
#goog-wm-qt:focus {
131+
border-color: #105cb6;
132+
outline: 0;
133+
color: #222;
134+
}
135+
136+
input::-moz-focus-inner {
137+
padding: 0;
138+
border: 0;
139+
}
140+
</style>
141+
</head>
142+
<body>
143+
<div class="container">
144+
<h1>Not found <span>:(</span></h1>
145+
<p>Sorry, but the page you were trying to view does not exist.</p>
146+
<p>It looks like this was the result of either:</p>
147+
<ul>
148+
<li>a mistyped address</li>
149+
<li>an out-of-date link</li>
150+
</ul>
151+
<script>
152+
var GOOG_FIXURL_LANG = (navigator.language || '').slice(0,2),GOOG_FIXURL_SITE = location.host;
153+
</script>
154+
<script src="http://linkhelp.clients.google.com/tbproxy/lh/wm/fixurl.js"></script>
155+
</div>
156+
</body>
157+
</html>

0 commit comments

Comments
 (0)