Skip to content

Commit be796e7

Browse files
gkdevopsgkdevops
authored
Revise README for clarity and content updates
Updated README to improve clarity and structure of topics. Merged JSON content into file handling section and added new sections for interview questions and projects.
1 parent cb6a815 commit be796e7

1 file changed

Lines changed: 54 additions & 63 deletions

File tree

‎README.md‎

Lines changed: 54 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
# Python Data Engineer Learning Repository
22

3-
Welcome to the **Python Data Engineer** learning repository! This repo contains a structured, practical set of Jupyter notebooks for learning core Python concepts, especially with a focus on data engineering. Each topic is covered with hands-on examples and explanations, and links are provided to the code for easy reference.
3+
Welcome to the **Python Data Engineer** learning repository! This repo contains a structured, practical set of Jupyter notebooks and example projects for learning core Python concepts with a focus on data engineering.
44

55
> **Note:** This summary is based on the top-level files; for a full list of all tutorials and scripts, check the [GitHub repository contents](https://github.com/gkdevops/python-data-engineer/tree/main/).
66
77
---
88

99
## 📚 Topics Covered
1010

11-
### 01. [Python Introduction](https://github.com/gkdevops/python-data-engineer/blob/main/1-Python-Introduction.ipynb)
11+
### 01. [Python Introduction](https://github.com/gkdevops/python-data-engineer/blob/main/01-Python-Introduction.ipynb)
1212
- **Overview:** Introduction to Python, variables, data types, and basic operations.
1313
- **Key Concepts:**
1414
- Printing and string manipulation
@@ -19,113 +19,104 @@ Welcome to the **Python Data Engineer** learning repository! This repo contains
1919

2020
---
2121

22-
### 02. [Python Conditions](https://github.com/gkdevops/python-data-engineer/blob/main/2-Python-Conditions.ipynb)
22+
### 02. [Python Conditions](https://github.com/gkdevops/python-data-engineer/blob/main/02-Python-Conditions.ipynb)
2323
- **Overview:** Mastering conditional statements for decision making.
24-
- **Key Concepts:**
25-
- `if`, `elif`, `else` statements
26-
- Comparison and logical operators
27-
- Nested conditions and practical examples
24+
- **Key Concepts:** `if`, `elif`, `else`, comparison and logical operators, nested conditions.
2825

2926
---
3027

31-
### 03. [Python Loops](https://github.com/gkdevops/python-data-engineer/blob/main/3-Python-Loops.ipynb)
28+
### 03. [Python Loops](https://github.com/gkdevops/python-data-engineer/blob/main/03-Python-Loops.ipynb)
3229
- **Overview:** Using loops to automate repetitive tasks.
33-
- **Key Concepts:**
34-
- `for` and `while` loops
35-
- Loop control (`break`, `continue`, `pass`)
36-
- Looping through lists, strings, and dictionaries
30+
- **Key Concepts:** `for` and `while` loops, loop control (`break`, `continue`, `pass`), iterating collections.
3731

3832
---
3933

40-
### 04. [Python Functions](https://github.com/gkdevops/python-data-engineer/blob/main/4-Python-Functions.ipynb)
34+
### 04. [Python Functions](https://github.com/gkdevops/python-data-engineer/blob/main/04-Python-Functions.ipynb)
4135
- **Overview:** Writing reusable blocks of code with functions.
42-
- **Key Concepts:**
43-
- Defining and calling functions
44-
- Parameters, return values, and scope
45-
- Lambda functions and higher-order functions
36+
- **Key Concepts:** Defining and calling functions, parameters, return values, scope, lambda functions.
4637

4738
---
4839

49-
### 05. [Python Operators](https://github.com/gkdevops/python-data-engineer/blob/main/5-Python-Operators.ipynb)
40+
### 05. [Python Operators](https://github.com/gkdevops/python-data-engineer/blob/main/05-Python-Operators.ipynb)
5041
- **Overview:** Using operators to manipulate data.
51-
- **Key Concepts:**
52-
- Arithmetic, assignment, comparison, logical, bitwise, and membership operators
53-
- Precedence and associativity
42+
- **Key Concepts:** Arithmetic, assignment, comparison, logical, bitwise, and membership operators.
5443

5544
---
5645

57-
### 06. [Python Collections](https://github.com/gkdevops/python-data-engineer/blob/main/6-Python-Collection.ipynb)
46+
### 06. [Python Collections](https://github.com/gkdevops/python-data-engineer/blob/main/06-Python-Collection.ipynb)
5847
- **Overview:** Mastering data structures for efficient storage and retrieval.
59-
- **Key Concepts:**
60-
- Lists, tuples, sets, dictionaries
61-
- When and how to use each collection
62-
- Real-world data engineering examples using collections
48+
- **Key Concepts:** Lists, tuples, sets, dictionaries, and real-world examples.
6349

6450
---
6551

66-
### 07. [Python Modules & Packages](https://github.com/gkdevops/python-data-engineer/blob/main/7-Python-Modules%20%26%20Packages.ipynb)
52+
### 07. [Python Modules & Packages](https://github.com/gkdevops/python-data-engineer/blob/main/07-Python-Modules%20%26%20Packages.ipynb)
6753
- **Overview:** Organizing and reusing code with modules and packages.
6854
- **Key Concepts:**
69-
- The difference between modules, packages, and libraries (with LEGO analogies)
70-
- Importing and using built-in and external libraries (e.g., Pandas, NumPy, Matplotlib, Requests, Scikit-learn)
55+
- Difference between modules, packages, and libraries
56+
- Importing and using built-in and external libraries (e.g., Pandas, NumPy, Matplotlib, Requests)
7157
- Creating custom modules and packages
7258

7359
---
7460

75-
### 08. [Files-CSV Directory](https://github.com/gkdevops/python-data-engineer/tree/main/files)
76-
- **Overview:** CSV File handling and manipulation for data storage and retrieval.
61+
### 08. [Files, CSV & JSON](https://github.com/gkdevops/python-data-engineer/tree/main/files)
62+
- **Overview:** File handling (text & CSV) and JSON management for configuration and data exchange. (JSON content previously listed separately has been merged into this section.)
7763
- **Key Concepts:**
78-
- Reading and writing text and CSV files
79-
- Using csv file with Pandas library
80-
- File and directory operations using `os` and `shutil`
81-
- Handling file paths and exceptions
82-
- Data extraction and ingestion from files
64+
- Reading and writing text and CSV files using built-in modules and pandas
65+
- Using `os` and `shutil` for file and directory operations
66+
- Reading, writing, parsing, and serializing JSON with Python’s `json` module
67+
- Data extraction and ingestion from files and JSON API responses
68+
- Error handling and path management
8369

8470
---
8571

86-
### 09. [JSON Directory](https://github.com/gkdevops/python-data-engineer/tree/main/json)
87-
- **Overview:** Managing JSON data formats for configuration and data exchange.
72+
### 09. [Python OOPs](https://github.com/gkdevops/python-data-engineer/tree/main/09-Python-OOPs)
73+
- **Overview:** Object-oriented programming in Python.
8874
- **Key Concepts:**
89-
- Reading and writing JSON files with Python’s `json` module
90-
- Parsing and serializing complex JSON structures
91-
- Real-world use cases: configuration files, API responses
92-
- Data transformation between JSON and Python objects
75+
- Defining classes and creating objects
76+
- Constructors (`__init__`)
9377

9478
---
9579

96-
### 10. [Randoms Directory](https://github.com/gkdevops/python-data-engineer/tree/main/randoms)
80+
### 10. [Python Randoms](https://github.com/gkdevops/python-data-engineer/tree/main/10-Python-Random)
9781
- **Overview:** Working with randomness, generating random numbers and data for testing and simulations.
98-
- **Key Concepts:**
99-
- Using Python’s `random` module for numbers, choices, and shuffling
100-
- Generating random data for data engineering tasks
101-
- Introduction to the `faker` library for synthetic data creation
102-
- Practical examples: random sampling, data anonymization
82+
- **Key Concepts:** `random` module, `faker` library, random sampling and anonymization.
10383

10484
---
10585

106-
### 11. [Blocks Directory](https://github.com/gkdevops/python-data-engineer/tree/main/blocks)
107-
- **Overview:** Code blocks and reusable scripts for modular data engineering workflows.
108-
- **Key Concepts:**
109-
- Encapsulating logic in code blocks (functions, scripts)
110-
- Organizing reusable code for ETL pipelines
111-
- Example templates for batch processing and automation
86+
### 11. [Code Blocks & Utilities](https://github.com/gkdevops/python-data-engineer/tree/main/11-Python-Blocks)
87+
- **Overview:** Reusable scripts and code blocks for modular data engineering workflows.
88+
- **Key Concepts:** Encapsulating logic in functions and scripts, templates for batch processing.
11289

11390
---
11491

115-
### 12. [Logging Directory](https://github.com/gkdevops/python-data-engineer/tree/main/logging)
92+
### 12. [Logging & Monitoring](https://github.com/gkdevops/python-data-engineer/tree/main/12-Python-Logging)
11693
- **Overview:** Logging and monitoring data engineering processes.
117-
- **Key Concepts:**
118-
- Using Python’s `logging` module for event tracking
119-
- Setting up log formats, levels, and handlers
120-
- Best practices for error handling and process monitoring
121-
- Writing logs to files and integrating with external tools
94+
- **Key Concepts:** Python’s `logging` module, log formats, levels, handlers, and best practices.
95+
96+
---
97+
98+
### 13. [Python Interview Qs](https://github.com/gkdevops/python-data-engineer/tree/main/13-Python-Interview-Qs)
99+
- **Overview:** Common Python interview questions and concise answers, focusing on practical explanations.
100+
101+
---
102+
103+
### 14. [Project: Streamlit Examples](https://github.com/gkdevops/python-data-engineer/tree/main/14-Project-Streamlit)
104+
- **Overview:** Example Streamlit apps and instructions to run them.
105+
- **Key Concepts:** Installing Streamlit, building and running simple apps, basic visualization with Plotly.
106+
107+
---
108+
109+
### 15. [Capstone Projects](https://github.com/gkdevops/python-data-engineer/tree/main/15-Capstone-Projects)
110+
- **Overview:** End-to-end projects to apply learned concepts.
111+
- **Contents:** Project ideas, example implementations, and deployment notes.
112+
122113
---
123114

124115
## 📎 How to Use This Repo
125116

126-
1. **Browse Notebooks:** Start with the Jupyter notebooks in the main directory for a structured learning path.
127-
2. **Explore Directories:** Check out the additional folders for more scripts and data.
128-
3. **Try the Code:** Run the notebooks locally or in an online Jupyter environment.
117+
1. **Browse Notebooks:** Start with the Jupyter notebooks in the main directory for a structured learning path.
118+
2. **Explore Directories:** Check out the additional folders for sample scripts, data, and projects.
119+
3. **Try the Code:** Run the notebooks locally or in an online Jupyter environment.
129120
4. **Contribute:** Pull requests to add new topics or improve examples are welcome!
130121

131122
---

0 commit comments

Comments
 (0)