Assignment

sdas1 · web-flow · commit f23f01965002 · 2021-05-11T05:34:18.000+02:00
diff --git a/README.md b/README.md
@@ -0,0 +1,82 @@
+# Homework 5 - Advanced NumPy
+
+## Deadline
+
+The deadline for this homework is on **Monday, 17th of May (2021-05-17 00:00:00 UTC+2)**.
+
+## New Homework System
+
+In response to the feedback we got through the *Homework System Survey* on StudIP, it is now only necessary to **solve 2/3 of the subtasks** in a homework order to **pass** a homework. However, we still highly encourage you to try to solve all tasks whenever possible. Solving a problem on your own is the only way you can really know if you have understood a concept.
+
+This change comes with a partly new set of commands that you can use:
+
+* Run `pytest test_[FILE-TO-TEST].py` to see if you have passed a subtask (e.g. `pytest test_omitting_outliers.py`)
+* Run `pytest` to see whether you have passed all subtasks - **this still works, but is not what determines the checkmark any more**
+* Run `python pass_check.py` to check how many points you currently have (one subtask will usually give 10 points) - **this is what determines the checkmark now**
+
+**The checkmark next to your commit on GitHub is still the deciding factor in whether you pass a homework or not.**
+
+If you encounter any problems with the new homework system, first check the forum. If it seems to be an unknown issue, please write to [Martin](mailto:mpoemsl@uos.de) so that it can be fixed or make a public post in the forum if it is a question whose answer might be of interest to others. 
+
+## This Homework
+
+This homework is about working with more advanced NumPy concepts. You will solve three tasks, one each in the following files:
+
+* Omitting Outliers: `omitting_outliers.py`
+* Rigorous Ranking: `rigorous_ranking.py`
+* Printing Primes: `printing_primes.py` 
+
+Template function headers are given in the files. You may use the full functionality of NumPy, but **no other libraries**.
+
+As always, you pass by pushing your solution to GitHub and having the green checkmark appear next to your commit. You can also ask our Telegram bot: `@uos_scipy_bot`. Running `python pass_check.py` on your own machine should also give the same result, but it's not what counts in the end!
+
+### 1 Omitting Outliers
+
+A common necessity in scientific programming is the removal of outliers, i.e. data points that were likely created by mistake. Write a function `remove_outliers` that takes in an n-dimensional (numeric) array `data` as a positional parameter, as well as keyword parameters `m` (default: `3`) and `replace` (default: `True`).
+
+Your function should detect every value as an outlier that is further than `m * sigma` away from the mean of `data`, where `sigma` is the standard deviation of `data`.
+
+ - If `replace` is `True`, the function should return an array of the same shape as `data`, where every outlier is replaced with the mean of the original input array.
+
+ - If `replace` is `False`, the function should return a 1D array containing all values of the input array except for the outliers.
+
+**Note:** For the purposes of this task, assume that mean and standard deviation may be calculated once in advance, even though at that point the outliers are still present in the dataset. There exist better methods for detecting outliers, but those are trickier to implement and beyond the scope of this task.
+
+**Functions that may be helpful:** `np.mean`, `np.std`, `np.abs`, `np.ndarray.copy`
+
+### 2 Rigorous Ranking
+
+Write a function `compute_rank` that takes a numeric array `array` of any shape as input and computes the **rank** of each value.
+
+The **rank** is an integer between 0 and `array.size - 1`. It signifies the position of each element in a *sorted* version of the array. For example: `[20, 30, -10]` should map to `[1, 2, 0]`.
+
+In the test examples, there will be no arrays that include ties. However, a good stance to take would be to give the element that comes first in the original array the lower rank.
+
+**Hint:** The solution to this assignment can potentially be very short, but that doesn't mean it's straightforward. This task will be easier if you have a good intuition for indexing with integer arrays. Additionally, it may be a good idea to think about 1D arrays first and then adapt your solution for more dimensions.
+
+**Functions that may be helpful:** `np.argsort`, `np.arange`, `np.flatten`
+
+### 3 Printing Primes
+
+For this task, you are given two helper functions that are already implemented:
+
+ - `is_prime(n)` will return True if `n` is a prime number, `False` otherwise
+ - `pretty_print_bool_array(array)` will print a boolean array to the console such that `True` is printed as "x" and `False` is printed as "."
+
+**Step 1:** Create a new function `is_prime_numpy`, that works just like `is_prime` but will accept an entire NumPy array as input and return the result element-wise.
+
+**Step 2:** Write a function `print_primes(rows, cols)`. Internally, it should create a boolean array of shape `(rows, cols)`. This array should be `True` exactly where the index is a prime number.
+
+How is this supposed to work with a 2D array, you ask? We have a row and a column index after all. What I mean is that you should imagine the values of the array being enumerated from top-left to bottom-right for this task, as in the example below:
+
+| **0**  | **1**  | **2**  | **3**  | **4**  |
+|:--:|:--:|:--:|:--:|:--:|
+| **5**  | **6**  | **7**  | **8**  | **9**  |
+| **10** | **11** | **12** | **13** | **14** |
+| **15** | **16** | **17** | **18** | **19** |
+
+It is up to you to find a suitable way to convert from `(row, col)` to this number.
+
+`print_primes` should then print this boolean array using the given pretty-print function. There should be no return value.
+
+**Functions that may be helpful:** `np.vectorize` and `np.indices` (only discussed in the Jupyter notebook)
diff --git a/helpers.py b/helpers.py
@@ -0,0 +1,44 @@
+import numpy as np
+
+import types
+
+def is_prime(n):
+    """simple function to check if a given integer is prime"""
+    # no change necessary here
+
+    if n <= 1:
+        return False
+
+    if n > 2 and n % 2 == 0:
+        return False
+
+    for i in range(3, int(np.sqrt(n)) + 1, 2):
+        if n % i == 0:
+            return False
+
+    return True
+
+
+def pretty_print_bool_array(array):
+    """this function will print a boolean array such that True values are 'x'
+    and False values are '.'"""
+
+    with np.printoptions(formatter={"bool": lambda b: "x" if b else "."}):
+        print(array)
+
+
+def imports_of_your_file(filename, testfile):
+    """ Yields all imports in the testfile. """
+
+    for name, val in vars(testfile).items():
+        if isinstance(val, types.ModuleType):
+            # get direct imports
+            yield val.__name__
+
+        else:
+            # get from x import y imports
+            imprt = getattr(testfile, name)
+
+            if hasattr(imprt, "__module__") and not str(imprt.__module__).startswith("_") and not str(imprt.__module__) == filename:
+                yield imprt.__module__
+
diff --git a/omitting_outliers.py b/omitting_outliers.py
@@ -0,0 +1,25 @@
+import numpy as np
+
+#def remove_outliers(data, m=3, replace=True):
+    # implement task 1 here
+    #raise NotImplementedError
+    
+def remove_outliers(an_array, m=3, replace=True):
+    mean = np.mean(an_array)
+    sigma = np.std(an_array)
+    distance_from_mean = abs(an_array - mean)
+    not_outlier = distance_from_mean < m * sigma
+    if (replace) :
+        return mean
+    else:
+        return an_array[not_outlier]
+    
+    
+if __name__ == "__main__":
+    # use this for your own testing!
+
+    data = np.array([10, 10, 10, 17, 10, 10])
+    print(remove_outliers(data, m=3, replace=False))
+    
+
+
diff --git a/pass_check.py b/pass_check.py
@@ -0,0 +1,49 @@
+from test_omitting_outliers import test_remove_outliers
+from test_rigorous_ranking import test_compute_rank
+from test_printing_primes import test_print_primes
+
+import pytest
+
+SUBTASKS = {
+    "Omitting Outliers": (test_remove_outliers, 10),
+    "Rigorous Ranking": (test_compute_rank, 10),
+    "Printing Primes": (test_print_primes, 10)
+}
+
+def test_entire_homework():
+
+    points_achieved = 0
+    points_possible = 0
+
+    print("Checking {} subtasks to see if you reached at least 2/3 of the possible points ...\n".format(len(SUBTASKS)))
+
+    for subtask_name, (subtask_test_function, subtask_points) in SUBTASKS.items():
+
+        points_possible += subtask_points
+
+        try:
+
+            subtask_test_function()
+
+            # this line will only be reached if no AssertionError or other Exception was thrown in test_function() 
+            points_achieved += subtask_points
+
+            print("Subtask '{}' with {} points: All good here!".format(subtask_name, subtask_points))
+
+        except Exception:
+
+            print("Subtask '{}' with {} points: Something went wrong. Better run 'pytest' on the corresponding pytest file to learn more!".format(subtask_name, subtask_points))
+        
+    if points_achieved >= (2/3) * points_possible:
+
+        print("\nCongratulations! You passed this homework with {} out of {} points.".format(points_achieved, points_possible))
+        print("Do not forget to commit and push the current state of your code - the pass is only official once you see the checkmark on GitHub!")
+        exit(0)
+
+    else:
+
+        print("\nSorry! You only achieved {} points which is less than 2/3 of the possible {} points.".format(points_achieved, points_possible))
+        exit(1)
+
+if __name__ == "__main__":
+    test_entire_homework()
diff --git a/printing_primes.py b/printing_primes.py
@@ -0,0 +1,10 @@
+from helpers import is_prime, pretty_print_bool_array
+
+import numpy as np
+
+# implement step 1 of task 3: printing primes here
+is_prime_numpy = None
+
+def print_primes(rows, cols):
+    # implement step 2 of task 3: prining primes here
+    raise NotImplementedError
diff --git a/requirements.txt b/requirements.txt
@@ -0,0 +1,2 @@
+pytest>=4
+numpy>=1.20
diff --git a/rigorous_ranking.py b/rigorous_ranking.py
@@ -0,0 +1,7 @@
+import numpy as np
+
+def compute_rank(array):
+
+    # implement task 2: rigorous ranking here
+    raise NotImplementedError
+
diff --git a/test_omitting_outliers.py b/test_omitting_outliers.py
@@ -0,0 +1,34 @@
+from helpers import imports_of_your_file
+
+from hashlib import sha1
+
+import numpy as np
+
+try:
+    import omitting_outliers as testfile
+except ModuleNotFoundError:
+    assert False, "The name of your file is supposed to be 'omitting_outliers.py'!"
+
+
+def test_remove_outliers(filename="omitting_outliers", allowed_imports={"numpy"}):
+    data = np.array([[-2,  1,  1,  0,  2, -1,  1, -3, -3, -2, -3, -3,-42,  1,  2,  1,  0,  1, -1, -1],
+                     [-2, -3,  2,  2, -3, 42,  0,  2,  0, -1, -3,  1,  2,  1, -3,  0,  0, -2,  0,  1]])
+
+    data = data.astype(float)
+
+    result = testfile.remove_outliers(data)
+
+    assert result.shape == data.shape, "If replace is True, your function should preserve the shape of the array"
+    assert sha1(result).hexdigest() == "52ffe05d09dbe4135e1a1a16e113a1a578775a56", "Your function does not seem to return the correct result"
+
+    result = testfile.remove_outliers(data, m=4.4)
+
+    assert sha1(result).hexdigest() == "15e350705c1a1d2b9a8cb16b0cf05b2266a9a2fd", "Your function does not seem to return the correct result"
+
+    result = testfile.remove_outliers(data, replace=False)
+
+    assert result.shape == (38, ), "If replace is False, your function should reduce the number of elements"
+    assert sha1(result).hexdigest() == "c51ef6b5de6a9654dcf2d4f69de324eb2591cac6", "Your function does not seem to return the correct result"
+
+    assert set(imports_of_your_file(filename, testfile)) <= allowed_imports, "You are not allowed to import any modules except NumPy!"
+
diff --git a/test_printing_primes.py b/test_printing_primes.py
@@ -0,0 +1,41 @@
+from helpers import imports_of_your_file
+
+from hashlib import sha1
+from io import StringIO
+
+import numpy as np
+
+import sys
+
+try:
+    import printing_primes as testfile
+except ModuleNotFoundError:
+    assert False, "The name of your file is supposed to be 'printing_primes.py'!"
+
+
+def test_print_primes(filename="printing_primes", allowed_imports={"numpy", "helpers"}):
+
+    assert callable(testfile.is_prime_numpy), "Your script does not have a function 'is_prime_numpy'"
+
+    try:
+        testfile.is_prime_numpy(np.arange(3))
+    except ValueError:
+        assert False, "Your 'is_prime_numpy' function does not seem to be able to handle NumPy arrays"
+
+    test_stdout = StringIO()
+    saved_stdout = sys.stdout
+
+    sys.stdout = test_stdout
+
+    result = testfile.print_primes(20, 20)
+
+    sys.stdout = saved_stdout
+
+    s = test_stdout.getvalue()
+
+    assert result is None, "Your function 'print_primes' should not have a return value"
+    assert len(s) > 0, "Your function 'print_primes' should *print* its result!"
+    assert sha1(s.encode("utf-8")).hexdigest() == "3c49338521ff33568de33bdeb89ae5b83a5b6c45", "Your function 'print_primes' does not seem to print the correct result"
+
+    assert set(imports_of_your_file(filename, testfile)) <= allowed_imports, "You are not allowed to import any modules except NumPy and helpers.py!"
+
diff --git a/test_rigorous_ranking.py b/test_rigorous_ranking.py
@@ -0,0 +1,21 @@
+from helpers import imports_of_your_file
+
+import numpy as np
+
+try:
+    import rigorous_ranking as testfile
+except ModuleNotFoundError:
+    assert False, "The name of your file is supposed to be 'rigorous_ranking.py'!"
+
+
+def test_compute_rank(filename="rigorous_ranking", allowed_imports={"numpy"}):
+
+    data = np.random.choice(100, size=(5, 5), replace=False)
+    rank = testfile.compute_rank(data)
+
+    assert rank.shape == data.shape, "The returned array does not have the same shape as the input array"
+    assert np.array_equal(np.sort(np.unique(rank)), np.arange(rank.size)), "The returned array contains invalid values for a rank"
+    assert np.array_equal(data.argsort(), rank.argsort()), "The returned array does not contain the correct ranks"
+
+    assert set(imports_of_your_file(filename, testfile)) <= allowed_imports, "You are not allowed to import any modules except NumPy!"
+
