TODO List for Experimental Mode Plus- GSOC 2013

This would also be a break down of my thought process and ideas as I tackle various tasks. Also lines are fairly long. Make sure you turn on word wrap. ;)

Manindra Moharana (me@mkmoharana.com)

* : Todo, x : Done, ? : Undecided Todo, ! : Critical, + : Minor Todo

Code Completion

===============

The big stuff:

*! May be I should just implement recursive find for compiled code first, see how it goes and hopefully it would give me some ideas about how to integrating the two.

- Making very good progress here. The elegance of recurion - Hats off!

- Many of the cases seem to have been covered, and I'm achieving more and more code unification as I'm working through the problem step by step

- Looks almost complete now, nearly all cases covered(July 13th)

x After popup appears, the popup location is fixed for the current line. So if editor window is moved while staying in the same line, popup appears at the prev location. Need to ensure editor is still at last know location. Fixed.

* Keyboard Shortcut for completion popup - Ctrl + Space

* Scope handling? Static/non static scope?

* Disable completions on comment line

* Trie implementation would be lower priority, "premature optimisation is pure evil". Get all features of CC working good enough and then plan this.

x Ensure that a compilation unit is created at startup!

x! Code competition for local code is working with recursive look up.

x Completion doesn't seem to show up for fields of a type defined locally. But works for methods with return type defined locally. Take ideas. Some case missing most probably. Fixed

x Discovered another major issue due to offset differences -> While looking for predictions, if the parsed string contains pde enhancements, predictions FAIL! Zomg.

Ex - "s.substring(int(13.4))." fails. Thinking to just do the substitutions before sending it to updatePredictions(), coz offsets aren't really a concern here, right? Yup, fixed it!

x! Code completion with library code, non-nested seems to be broken, fix it. Fixed.

x Completion for external classes - ArrayList, HashMap, etc.

x! Recursive lookup for compiled(library) code!

x! Library CC for nested would be tricky. Need to jump from local->compiled code while searching recursively. Recursive find's current implementation is based on ASTNode return type. Afaik, no way to instantiate orphaned ASTNode objects(or did I miss it?). ASTNode objects have to be created only from the main ast instance. But I need to find a way to switch to compiled instances from local class instance.

x! Should I implement wrapper for ASTNode? - possibly needed for code completion with compiled and non-compiled code. Done.

x Differentiating between multiple statements on the same line. How to? Done with offset handling.

x - Cache predictions if current 'word' is increasing in length. If already showing predictions beginning with 's', for 'sa', remove extra completions, rather than recalculating predictions. Performance increase.

x Parameterized type support is broken.

x Array types, all all other types support broken. :\

x Completion for array access, strings[0].

Finer details

* findDeclarations should support 3rd party classes too. It's about time. ;)

* printStuff(int,float,String) - completion endings have to be appropriate. Right now it's just printStuff(,,). Cursor positioning also needs to be taken care of(done). Argument list as tooltip if possible?

*! p5 enhanced stuff in java, how does it fit in with everything else, and edge cases. Possibly add support for them. Offset handling improvements should help here.

* Diamond operator isn't supported for now. Bummer.

x Completion popup height is now dynamic, decreases to fit.

* Completion width can be dynamic, if really needed..

x Icons for completions? Or overkill right now?

x 'Show Usage' menu item added

x Show declaring class for completions

x! Ignore String case while finding completion candidates

x Multiple 3rd party classes found in various packages. Not a chance no more.

x Obj a1; a1.-> completion doesn't work before it is instantiated. Look into that. Began working again by itself. Yay!

x Cursor positioning should be after the first ( if arguments present, else after ()

x Display the type of Completion(method return type, variable type) in the popup.

- facing some issues for local types. Fixed.

x Sorted list of completion candidates - fields, then methods. It's unsorted presently.

x Reflection API - getMethods vs getDeclaredMethods. declared.

x Need to add offset correction to ASTGenerator and its lookup methods. Or leave it for later? All set to implement

x Completion List should get hidden on hitting esc key

Offset Mapping

==============

First major hurdle is offset mapping

*! pde<->java code offset : precise conversion needed

* W.r.t PDE specific enhancements, things are almost working. There are some offset issues when multiple pde statements are in the same line, but I guess it's good enough for now to proceed ahead. Will keep a close watch for potential bugs.

x for the above, I've decide to first implement a sketch outline like feature, which would highlight an AST element precisely in the pde code. This would ensure I've got the mapping working properly. And may lead to a future feature.

x This is precise upto a certain line. Once on a line, pde stuff have to be taken into consideration.

x Edge case - multiple statements in a single line

x PDE specific enhancements will also have to be tackled like int(), # literals. The length of the node returned needs to be modified to make up for extra chars added like PApplet.parseFloat, etc. Also the 2nd or futher pde enhancements in the same line means even the beginning offset would need adjustment. Meh.

Refactoring

===========

* Undo misbehaves here, handle carefully.

* Fails to rename the first defined global variable, if a javadoc comment precedes it. But owrds for single/multiline comments. Wth!

x New Name is validated.

x Ordered list in 'Show Usage' window

x Add support for word select on right click and rename, mouse co-ordinates need to obtained carefully

Refactoring would work only when code is compiler error free. I plan to do a find replace type op on the compile ready code.

1. First identify the declaration of the variable in the AST. We'll then make a list of all its occurrences.

2. DFS through the AST, for each (SimpleName)instance of the word in code, find if the matched word is the same one whose declaration we found.

x Edge Case: For renaming a TypeDeclaration, the declaration of SimpleName instance of the TD and it's constructor(s) aren't added to the list generated by DFS. So for renaming TD, will have to manually add the TD SimpleName and it's constructors' SimpleNames to the a list of declaration nodes that can be positively matched against.

x Renaming any constructor is equivalent to renaming the TD

3. Find corresponding PDE offsets of the SimpleNames, rename in each line.

x Edge Case: Need to take displaced offsets on a line, due to pde enhancements, into consideration.

4. All the changes in code would be made in a separate copy of the code(?). After all the renaming is done, allow it only if the new code compiles. Basically an undo should be possible in case of conflicts.

x Refactoring ui

x For now, user needs to highlight the name of the var, and then right-click -> Rename..

x Handle saving. If sketch closed after renaming w/o saving find bugs. Done, marking the sketch as modified after renaming.

Quick Navigation

================

*+ A silly bug where the name of the first field declaration isn't highlighted correctly. Seems to be happening if there's a javadoc or multiline comment near about the top.

x On OS X, Ctrl + Click is right mouse click, so implement Cmd + Click instead. isMetaDown()?

x Ctrl + Click on an element to scroll to its definition in code

x Local Vars

x Local Methods

x Local Classes

x Recursive lookup, a.b().c()

x Now highlihgting the declaration name, rather than the whole declaration.

Sketch Outline

==============

x Show Sketch Outline Tree

x Filter stuff in text field

x Add icons - custom cell renderer

Suggestion for missing imports

==============================

* Find a more subtle way to suggest for imports. The current method is too troublesome. Randomly pops up offering suggestions. May intimidate beginners.

1. In compileCheck() in ECS, check if error message is of the type "__" cannot be resolved to a type.

2. Find the class name via astGen, and suggest import as a popup.

x Barebones functionality done.

x Add imports only to beginning of first tab.

x Search within contributed libraries folder

x Hide suggestion list before showing import suggestions

x Search within code folder of sketch

Labels for Java elements

========================

x Working for local code

* Need to modify getASTNodeAt to also fetch the type for predefined classes.

* Labels for predefined class objects

* Chaining support for labels

Synchronization

===============

Gotta do it carefully between main thread, ECS Thread, and SwingWorker threads

Fields that are concurrently accessed:

ECS members:

x ArrayList<Problem> problems - updated in ECS, accessed by ErrorBar.update()

x ArrayList<URL> classpathJars - updated in ECS, accessed by ASTGenerator.loadJars()

x hasErrors, syntaxErrors - Atomic Boolean

x boolean warningsEnabled - made it volatile

* CompilationUnit cu - updated in ECS, accessed a zillion times in ASTGenerator :'(

General Stuff

=============

* [Critical] PermGen out of memory bug. Manually triggering GC after making the classloader null ensures permgen memory is reclaimed on editor exit. Max open window still limited by max permgen size. Also, added a classloadcounter in ECS to trigger GC periodically.

https://github.com/processing/processing-experimental/issues/1

See: http://stackoverflow.com/questions/2095974/how-to-unload-a-already-loaded-class-in-java

I'm making the classLoader null, but what about the classes loaded by ASTGen? Investigate.

x Disabling Error Checking disables predictions as well! Fixed.

x Added doc listener for text area updates

x Consult Ben on where to save preferences - main preferences.txt or custom one. - Main prefs file

x Save preferences to main preference.txt

x Hide breakpoint markers when Debugger isn't active

x Ensure gutter mouse handler is taken care of when hiding Debugger breakpoint bar.

x Ensure all editor windows are closed when editor is closed.

x Add a red marker near Errors label in console toggle, to indicate errors present in sketch.

x Add option for toggling debug output

x On Run/Debug Console is visible(ProblemsList hidden)

* Update wiki for Ctrl + H instead of Ctrl + J shortcuts

x update build.xml to produce dists

x Make this a contributed mode - mode.txt, github releases feature, version numbering, git tags, etc

x Add GitHub link to PDE X Menu