forth: evaluateBoth property #2018

Activity

1. glennj commented on May 5, 2022

   

   glennj

   on May 5, 2022

   Contributor

   More actions

   Nobody's commented on this so far: my take:

   The README describes it like

      " The 'property' is a string in lowerCamelCase identifying ",
      " the type of test, but most of the times it is just the   ",
      " name of a function being tested.                         ",
   

   I don't see it documented anywhere else as meaning anything more specific.

   Clearly, it can't be absolutely relied upon to be anything more than a label describing some action to impose upon the system under test.

   For forth specifically, the point is to instantiate two forth evaluators and ensure that inputs to one do not bleed into the other -- that can't be a single function on a single instance. (The bash track excludes this test: clearly two separate scripts running in different processes cannot accidentally interact).

   Sadly, the generator will need some specific instructions for some exercises.

   👍React with 👍2Reacted by Norbert Melzer and Erik Schierboom
2. ErikSchierboom commented on May 6, 2022

   

   ErikSchierboom

   on May 6, 2022

   Member

   More actions

   I don't see it documented anywhere else as meaning anything more specific.

   Clearly, it can't be absolutely relied upon to be anything more than a label describing some action to impose upon the system under test.

   This is true. In 99% of the test cases, the property can be used as the function name, but there are test cases where this can't work, either because the thing that is being tested is impossible to map to a JSON object with inputs/outputs (e.g. something being random, see https://github.com/exercism/problem-specifications/blob/main/exercises/diffie-hellman/canonical-data.json#L33), or because it is describing multiple actions (like the forth test case you mentioned).

   Sadly, the generator will need some specific instructions for some exercises.

   It should indeed. I wonder if it would be a good idea to add a scenario for these exceptional test cases, which could then be detected by a generator.
3. kytrinyx commented on Nov 10, 2022

   

   kytrinyx

   on Nov 10, 2022

   Member

   More actions

   I think a scenario is a very good suggestion.
   How about local-scope? We already have that scenario, and it seems to fit the bill.
4. ErikSchierboom commented on Nov 11, 2022

   

   ErikSchierboom

   on Nov 11, 2022

   Last edited by ErikSchierboom

   Member

   More actions

   How about local-scope? We already have that scenario, and it seems to fit the bill.

   Yes, that does indeed fit the bill well and should indeed be added!

   What I was originally suggesting was to add another scenario, namely one that indicates that the property is different from a regular property and requires special treatment by a test generator. But maybe we don't need it as certain scopes will always require this special treatment, like local-scope and immutable. Maybe it is enough to identify all test cases that have special behavior and a add an appropriate scenario?

   Here is another weird test case in dnd-character:

   {
     "uuid": "2ca77b9b-c099-46c3-a02c-0d0f68ffa0fe",
     "description": "each ability is only calculated once",
     "property": "strength",
     "input": {},
     "expected": "strength == strength"
   }

   Should this have a memoized or memoization scenario?
5. kytrinyx commented on Nov 11, 2022

   

   kytrinyx

   on Nov 11, 2022

   Member

   More actions

   Ah, yeah, memoization is probably a good idea there.