Skip to content

Commit a24f885

Browse files
jvmvikjvmvik
committed
add better documentation system
1 parent 2d59f01 commit a24f885

File tree

7 files changed

+305
-21
lines changed

7 files changed

+305
-21
lines changed

‎.gitignore‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,4 +6,5 @@ serpapi.egg-info/
66

77
serpapi/__pycache__
88
tests/__pycache__
9-
*.pyc
9+
*.pyc
10+
.DS_Store

‎Makefile‎

Lines changed: 26 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -2,48 +2,55 @@
22
#
33
# current version
44
version=$(shell grep version setup.py | cut -d"'" -f2)
5+
dist=dist/serpapi-$(version).tar.gz
56

67
.PHONY: build
78

8-
all: clean install test test2
9+
all: clean install readme doc lint test build
910

1011
clean:
1112
find . -name '*.pyc' -delete
1213
find . -type d -name "__pycache__" -delete
13-
pip3 uninstall google_search_results
14-
15-
install:
16-
pip3 install -r requirements.txt
14+
pip3 uninstall serpapi
1715

16+
# lint check
1817
lint:
1918
pylint serpapi
2019

21-
# Test with Python 3
22-
test: lint
20+
# test with Python 3
21+
test:
2322
pytest --cov=serpapi tests/
2423

25-
# run example only
26-
# and display output (-s)
27-
example:
28-
pytest -s "tests/test_example.py::TestExample::test_async"
29-
24+
# pytest-cov - code coverage extension for pytest
25+
# sphinx - documentation
3026
install:
3127
pip3 install -U setuptools
32-
pip install pytest-cov
28+
pip3 install -r requirements.txt
29+
pip3 install pytest-cov
30+
pip3 install sphinx
31+
32+
readme:
33+
erb -T '-' README.md.erb > README.md
3334

34-
doc:
35-
pydoc
35+
doc: readme
36+
$(MAKE) -C docs/ html
3637

3738
# https://packaging.python.org/tutorials/packaging-projects/
38-
build: doc
39+
build: doc test
3940
python3 setup.py sdist
4041

42+
# out of box testing / user acceptance before delivery
4143
oobt: build
42-
pip3 install ./dist/google_search_results-$(version).tar.gz
44+
pip3 install ./${dist}
4345
python3 oobt/oobt.py
4446

4547
check: oobt
46-
twine check dist/google_search_results-$(version).tar.gz
48+
twine check ${dist}
4749

4850
release: # check
49-
twine upload dist/google_search_results-$(version).tar.gz
51+
twine upload ${dist}
52+
53+
# run example only
54+
# and display output (-s)
55+
example:
56+
pytest -s "tests/test_example.py::TestExample::test_async"

‎README.md‎

Lines changed: 85 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,85 @@
1-
master branch ready for pull request
1+
# User guide
2+
Scrape Google and other search engines from our fast, easy, and complete API using SerpApi.com
3+
This ruby library is meant to scrape and parse results from all major search engine available world wide including Google, Bing, Baidu, Yandex, Yahoo, Ebay, Apple and more using [SerpApi](https://serpapi.com).
4+
SerpApi.com provides a [script builder](https://serpapi.com/demo) to get you started quickly.
5+
6+
## Installation
7+
serpapi can be installed with pip.
8+
9+
```sh
10+
$ python -m pip install serpapi
11+
```
12+
13+
## Quick start
14+
First things first, import the serpapi module:
15+
16+
```python
17+
>>> import serpapi
18+
```
19+
You’ll need a Client instance to make search. This object handles all of the details of connection pooling and thread safety so that you don’t have to:
20+
21+
```python
22+
>>> client = serpapi.Client()
23+
```
24+
To make a search using SerpApi.com client.
25+
26+
```python
27+
>>> parameter = {
28+
api_key: "secret_api_key", # from serpapi.com
29+
engine: "google", # search engine
30+
q: "coffee", # search topic
31+
location: "Austin,TX" # location
32+
}
33+
results = searpapi.search(parameter)
34+
```
35+
Putting everything together.
36+
```python
37+
import serpapi
38+
39+
parameter = {
40+
api_key: "secret_api_key", # from serpapi.com
41+
engine: "google", # search engine
42+
q: "coffee", # search topic
43+
location: "Austin,TX" # location
44+
}
45+
results = searpapi.search(parameter)
46+
print(results)
47+
```
48+
49+
### Advanced settings
50+
SerpApi Client uses urllib3 under the hood.
51+
The HTTP connection be tuned by setting the following client specific setting.
52+
- retries : attempt to reconnect if the connection failed by default: False
53+
- timeout : connection timeout by default 60s
54+
for more details: https://urllib3.readthedocs.io/en/stable/user-guide.html
55+
56+
For example:
57+
parameter = {
58+
retries: 5,
59+
timeout: 4.0
60+
}
61+
62+
## Developer's note
63+
### Key goals
64+
- Brand centric instead of search engine based
65+
- No hard coded logic per search engine
66+
- Simple HTTP client (lightweight, reduced dependency)
67+
- No magic default values
68+
- Thread safe
69+
- Easy to extends
70+
- Defensive code style (raise cutsom exception)
71+
- TDD
72+
- Best API coding pratice per platform
73+
74+
### Design
75+
The API design was inpired by the most popular Python packages.
76+
- urllib3 - https://github.com/urllib3/urllib3
77+
- Boto3 - https://github.com/boto/boto3
78+
79+
### Quality expectation
80+
- 0 lint issues using pylint `make lint`
81+
- 99% code coverage running `make test`
82+
83+
### TODO
84+
- Add more test
85+
- Release flow

‎README.md.erb‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
<%-
2+
def snippet(format, path, start, stop)
3+
slice = File.new(path).readlines[start..stop]
4+
slice.reject! { |l| l =~ /expect\(/ }
5+
%Q(```#{format}\n#{slice.join}```\n see: #{path})
6+
end
7+
-%>
8+
# User guide
9+
Scrape Google and other search engines from our fast, easy, and complete API using SerpApi.com
10+
This ruby library is meant to scrape and parse results from all major search engine available world wide including Google, Bing, Baidu, Yandex, Yahoo, Ebay, Apple and more using [SerpApi](https://serpapi.com).
11+
SerpApi.com provides a [script builder](https://serpapi.com/demo) to get you started quickly.
12+
13+
## Installation
14+
serpapi can be installed with pip.
15+
16+
```sh
17+
$ python -m pip install serpapi
18+
```
19+
20+
## Quick start
21+
First things first, import the serpapi module:
22+
23+
```python
24+
>>> import serpapi
25+
```
26+
You’ll need a Client instance to make search. This object handles all of the details of connection pooling and thread safety so that you don’t have to:
27+
28+
```python
29+
>>> client = serpapi.Client()
30+
```
31+
To make a search using SerpApi.com client.
32+
33+
```python
34+
>>> parameter = {
35+
api_key: "secret_api_key", # from serpapi.com
36+
engine: "google", # search engine
37+
q: "coffee", # search topic
38+
location: "Austin,TX" # location
39+
}
40+
results = searpapi.search(parameter)
41+
```
42+
Putting everything together.
43+
```python
44+
import serpapi
45+
46+
parameter = {
47+
api_key: "secret_api_key", # from serpapi.com
48+
engine: "google", # search engine
49+
q: "coffee", # search topic
50+
location: "Austin,TX" # location
51+
}
52+
results = searpapi.search(parameter)
53+
print(results)
54+
```
55+
56+
### Advanced settings
57+
SerpApi Client uses urllib3 under the hood.
58+
The HTTP connection be tuned by setting the following client specific setting.
59+
- retries : attempt to reconnect if the connection failed by default: False
60+
- timeout : connection timeout by default 60s
61+
for more details: https://urllib3.readthedocs.io/en/stable/user-guide.html
62+
63+
For example:
64+
parameter = {
65+
retries: 5,
66+
timeout: 4.0
67+
}
68+
69+
## Developer's note
70+
### Key goals
71+
- Brand centric instead of search engine based
72+
- No hard coded logic per search engine
73+
- Simple HTTP client (lightweight, reduced dependency)
74+
- No magic default values
75+
- Thread safe
76+
- Easy to extends
77+
- Defensive code style (raise cutsom exception)
78+
- TDD
79+
- Best API coding pratice per platform
80+
81+
### Design
82+
The API design was inpired by the most popular Python packages.
83+
- urllib3 - https://github.com/urllib3/urllib3
84+
- Boto3 - https://github.com/boto/boto3
85+
86+
### Quality expectation
87+
- 0 lint issues using pylint `make lint`
88+
- 99% code coverage running `make test`

‎docs/Makefile‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Minimal makefile for Sphinx documentation
2+
#
3+
4+
# You can set these variables from the command line, and also
5+
# from the environment for the first two.
6+
SPHINXOPTS ?=
7+
SPHINXBUILD ?= sphinx-build
8+
SOURCEDIR = source
9+
BUILDDIR = build
10+
11+
# Put it first so that "make" without argument is like "make help".
12+
help:
13+
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
14+
15+
.PHONY: help Makefile
16+
17+
# Catch-all target: route all unknown targets to Sphinx using the new
18+
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
19+
%: Makefile
20+
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

‎docs/source/conf.py‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
# Configuration file for the Sphinx documentation builder.
2+
#
3+
# This file only contains a selection of the most common options. For a full
4+
# list see the documentation:
5+
# https://www.sphinx-doc.org/en/master/usage/configuration.html
6+
7+
# -- Path setup --------------------------------------------------------------
8+
9+
# If extensions (or modules to document with autodoc) are in another directory,
10+
# add these directories to sys.path here. If the directory is relative to the
11+
# documentation root, use os.path.abspath to make it absolute, like shown here.
12+
#
13+
import os
14+
import sys
15+
sys.path.insert(0, os.path.abspath('..'))
16+
17+
18+
# -- Project information -----------------------------------------------------
19+
20+
project = 'serapi-python'
21+
copyright = '2022, Victor Benarbia'
22+
author = 'Victor Benarbia'
23+
24+
# The full version, including alpha/beta/rc tags
25+
release = '1.0.0'
26+
27+
28+
# -- General configuration ---------------------------------------------------
29+
30+
# Add any Sphinx extension module names here, as strings. They can be
31+
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
32+
# ones.
33+
extensions = [
34+
'sphinx.ext.githubpages',
35+
'sphinx.ext.autodoc'
36+
]
37+
38+
# Add any paths that contain templates here, relative to this directory.
39+
templates_path = ['_templates']
40+
41+
# List of patterns, relative to source directory, that match files and
42+
# directories to ignore when looking for source files.
43+
# This pattern also affects html_static_path and html_extra_path.
44+
exclude_patterns = []
45+
46+
47+
# -- Options for HTML output -------------------------------------------------
48+
49+
# The theme to use for HTML and HTML Help pages. See the documentation for
50+
# a list of builtin themes.
51+
#
52+
html_theme = 'classic'
53+
54+
# Add any paths that contain custom static files (such as style sheets) here,
55+
# relative to this directory. They are copied after the builtin static files,
56+
# so a file named "default.css" will overwrite the builtin "default.css".
57+
html_static_path = ['_static']
58+
59+
60+
# -- Extension configuration -------------------------------------------------

‎docs/source/index.rst‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
.. serapi-python documentation master file, created by
2+
sphinx-quickstart on Sun Apr 3 21:09:40 2022.
3+
You can adapt this file completely to your liking, but it should at least
4+
contain the root `toctree` directive.
5+
6+
Welcome to serapi-python's documentation!
7+
=========================================
8+
9+
.. automodule:: serpapi
10+
:members: serpapi
11+
.. automodule:: serpapi.serpapi
12+
:members:
13+
.. toctree::
14+
:maxdepth: 2
15+
:caption: Contents:
16+
17+
18+
19+
Indices and tables
20+
==================
21+
22+
* :ref:`genindex`
23+
* :ref:`modindex`
24+
* :ref:`search`

0 commit comments

Comments
 (0)