Skip to content

DOC: plotting section for Users Guide#29124

Open
jklymak wants to merge 4 commits intomatplotlib:mainfrom
jklymak:doc-plotting
Open

DOC: plotting section for Users Guide#29124
jklymak wants to merge 4 commits intomatplotlib:mainfrom
jklymak:doc-plotting

Conversation

@jklymak
Copy link
Copy Markdown
Member

@jklymak jklymak commented Nov 11, 2024

This PR adds a "plotting" section to the Users Guide. See https://output.circle-artifacts.com/output/job/78379cd7-923b-4f4e-90ed-e297cace49c9/artifacts/0/doc/build/html/users/index.html (update 20 April) It is unusual to have a Users Guide that does not at least cover the basic ways to make visualizations.

The organization is the same as the Plot Types gallery, with the exception that I moved errorbar out of statistics to pairwise; everything else in "statistics" calculates statistics, errorbar, like fill_between/x just plots an idea of the errors given calculated errors.

This allows related visualization types to be shown in a curated order, with narrative connection between the topics. Currently it is pretty high level, and then details referred to the Gallery Examples as either links or in .. seealso:: callouts. Some more details could be covered in each of these sections over time.

I've not made any changes outside of these documents, except I added soft references so gallery items can be referenced by :ref: instead of :doc:. Probably the Gallery Examples could/should link back to the plotting sections. Possibly some gallery sections would want to be removed, but overall I think overlap between the presentations methods is fine.

In terms of adding/moving material the balance between whether information belongs in the Example or this section will require some back and forth. Overall I'd recommend erring on the side of redundancy, and using liberal cross linking.

@github-actions github-actions bot added Documentation: plot types files in galleries/plot_types Documentation: examples files in galleries/examples Documentation: user guide files in galleries/users_explain or doc/users labels Nov 11, 2024
@story645
Copy link
Copy Markdown
Member

story645 commented Nov 12, 2024

Given that this is clearly a lot of work, what's the motivation for this section/this new set of docs? What problem/questions from folks are you seeing that you're trying to solve here that's not currently being addressed in the gallery?

@jklymak jklymak force-pushed the doc-plotting branch 2 times, most recently from 235b82e to 4856e0d Compare November 12, 2024 18:01
@jklymak jklymak changed the title Doc plotting DOC: plotting section for Users Guide Nov 12, 2024
@jklymak jklymak marked this pull request as ready for review November 12, 2024 19:14
@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Jan 22, 2025

I'll ping for a review for this. Obviously this can just be considered scaffolding.

I believe I addressed why the gallery is not a replacement for explaining how to make plots in the User Guide in the PR description. The gallery will never be a cohesive introductory guide, and I think that is fine. The approach here is to provide a basic start with links to more examples in the appropriate section of the gallery, or elsewhere in the User Guide.

Copy link
Copy Markdown
Member

@timhoffm timhoffm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not having the time for a thorough review right now. I'll give a very brief feedback so that this does not stand uncommented.

I understand and agree with the fundamental idea. I'm slightly annoyed by the structural duplication of this page and "Plot types".

Way of working: This is a massive PR, which makes reviewing hard. I would have to reserve a significant time slot to look into it, which raises the bar and lowers the motivation. Additionally, the whole thing came without prior heads/up discussion. This means I first have to review overall approach and structure plus all the content intermixed and at the same time. I'm feeling overwhelmed by this. Also, I'm afraid of looking into/starting a discussion on the structure, because that may involve a lot of follow-up work due to all the details already written out.

Overall, this was the motivation for me to not look into a review.

IMHO this would work better if

  • you first create an issue describing the topic, and the idea for the stuctural solution (optionally/alternatively make a proof-of-concept PR for the structure). We could then agree on a the structure first.
  • then make a PR that puts the desired structure in place
  • after that, add multiple PRs for the individual sections.

This would keep the PRs smaller and more focused and significantly ease reviewing.

Comment thread galleries/examples/images_contours_and_fields/contour_demo.py
@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Jan 22, 2025

I'm slightly annoyed by the structural duplication of this page and "Plot types".

This was deliberately done so that they would present material in the same order but with more depth than in the quick visual index of "Plot Types". I'm not sure I understand the annoyance - particularly given that I made the Plot Types PR.

MHO this would work better if

you first create an issue describing the topic, and the idea for the stuctural solution (optionally/alternatively make a proof-> of-concept PR for the structure). We could then agree on a the structure first.
then make a PR that puts the desired structure in place
after that, add multiple PRs for the individual sections.
This would keep the PRs smaller and more focused and significantly ease reviewing.

I'd consider this the proof of concept PR. If you have serious objections to the structure, folks could propose alternates. If not, I'd suggest this PR is better than nothing, which is what we currently have. If folks disagree and feel it is worse than nothing, then we should close the PR and continue to have a large gap in the docs.

@story645
Copy link
Copy Markdown
Member

If folks disagree and feel it is worse than nothing, then we should close the PR and continue to have a large gap in the docs.

I'm not sure what gap it is you're trying to fill with these docs. The plot type gallery is already heavily curated, so I don't think that the narrative is adding much here.

Some of the examples feel so bogged down with extras that I'm not sure that readers will pick up the pattern that I think is what you're trying to convey, and information on how to use each specific function.

I think if there's a gap here, it's in understanding the artists that back the plots and I'd orient the user guide plotting section around those underlying artists.

@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Jan 22, 2025

The User guide has how to make a figure, how to make an axes, ...., a bunch of complicated stuff. I don't think "go look at the plot types gallery" is a good substitute for that middle ground, and I don't think the plot types gallery ties visualizations together in a hierarchy other than their ordering. Which is great - I look at the plot type gallery as a quick overview of what the library can do that a user can quickly scan to see if their chosen visualization is there, and see what we call it.

This new section of the User Guide should get into more details and connect the visualizations which is what I tried to start here. For sure, adding more detail about the underlying artists could be helpful, though I'd perhaps not ruin the flow so much as show a few examples of using the artists, and link out to the API docs or explanation elsewhere in the the Guide.

Some of the examples feel so bogged down with extras

I'm not sure what you are referring to here, but the examples are meant to be semi complete visualizations in most cases. If folks feel something got too far into the weeds, they are welcome to propose scaling a section back or breaking it into components.

I do somewhat feel you are expressing two contradictory ideas of what this section is meant for; "more details about underlying artists" and "fewer extras" are pulling in opposite directions. Obviously writing examples is a balancing act, and folks should feel free to edit this first pass all they want.

@story645
Copy link
Copy Markdown
Member

story645 commented Jan 22, 2025

The User guide has how to make a figure, how to make an axes, ...., a bunch of complicated stuff.

That's because the user guide introduces the concept of a figure, axes, colormap, etc, all of which are backed by objects. There's not a concept of a plot in the same manner, the closest being the artists that underlie the plot. And it covers the basics of how to make a plot in getting started and in the discussions of the library components relevant for that plot.

I'm not sure what you are referring to here, but the examples are meant to be semi complete visualizations in most cases.

For example, this gets so bogged down in the user warning (which is better explained in the API docs) that it loses focus on how to use the function (if that's the goal)
https://github.com/matplotlib/matplotlib/blob/2a0455e40a990bec76f671ef2d9be0cf7f6ea4af/galleries/users_explain/plotting/gridded.py#L95C5-L118

For example, the focus here is on setting up the data, the actual usage of the function not changing from the usage above:

def midpoints_plus_ends(x):
"""Given a 1D array, return a new array with the midpoints and the two ends."""
return np.concatenate(([x[0] - (x[1] - x[0]) / 2],
x[:-1] + np.diff(x) / 2,
[x[-1] - (x[-1] - x[-2]) / 2]))
yn = midpoints_plus_ends(y)
xn = midpoints_plus_ends(x)
Xn, Yn = np.meshgrid(xn, yn)
Yn = Yn + 0.3 * np.abs(xn)
fig, ax = plt.subplots()
ax.pcolormesh(Xn, Yn, Z, vmin=-0.5, vmax=1.0)

For example, the goal of this example is presumably to contrast the two, which a reader can pick up from the gallery thumbnails:

https://github.com/matplotlib/matplotlib/blob/2a0455e40a990bec76f671ef2d9be0cf7f6ea4af/galleries/users_explain/plotting/gridded.py#L150C5-L173

I do somewhat feel you are expressing two contradictory ideas of what this section is meant for; "more details about underlying artists" and "fewer extras" are pulling in opposite directions.

What I mean by focusing on the artist is more documents in the style of https://matplotlib.org/devdocs/users/explain/artists/imshow_extent.html. It unpacks a characteristic of ImageBase objects that once the reader understands it, they can then apply that understanding to every plotting method that takes an extent keyword.

@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Jan 23, 2025

That's because the user guide introduces the concept of a figure, axes, colormap, etc, all of which are backed by objects. There's not a concept of a plot in the same manner, the closest being the artists that underlie the plot. And it covers the basics of how to make a plot in getting started and in the discussions of the library components relevant for that plot.

From my point of view people need to be able to make a figure, make and axes, and then make visualizations in those figures and axes. The User Guide as it stands does not have that last step. They are all underscored by objects, so I'm not following what that has to do with explaining how to use the library.

For example, this gets so bogged down in the user warning (which is better explained in the API docs) that it loses focus on how to use the function (if that's the goal)

What I mean by focusing on the artist is more documents in the style of https://matplotlib.org/devdocs/users/explain/artists/imshow_extent.html.

I'd say both of these are somewhat asides that help the user better use the library. You could easily argue that the asides should be in a separate page or put in-line. But I don't think a deep-dive on every artist type and its quirks is the right level for the top-section of the user guide. From my point of view, if it can be explained quickly, inline is OK, if not then it should be linked out to a dedicated subpage.

@timhoffm
Copy link
Copy Markdown
Member

I give one high-level comment, other than that I'm out of the details here, because the PR gives me a mental overload:

I definitively see a place for a more detailed usage description of (some) plotting functions.
Examples:

  • plot() needs explanition at least on (i) plotting lines and/or markers, (ii) the use of the shorthand fmt string plot(x, y, 'r-') vs. keyword arguments; (iii) an explantion that and why the result is a list of Line2D and not a single Line2D plus the tuple-unpacking trick to put it into a single variable.
  • imshow() should have a basic explanation how to use color limits and colormaps for simple cases (not going deep dive into the data->color pipeline).

The user guide is certianly the right place for this. (How to plot with lines / markers / images). I'm not clear though on the framing / perspective. I'm feeling that the focus on "plotting methods" (Document: Using matplotlib > Plotting methods, and using method names as the sections) is wrong for a user guide. I think it should instead be something like "Using matplotlib > Visualizing different kinds of data". Then the subsections "pairwise data", "statistical plots", ... in here are ok. But within these sections again, it should not be methods but "drawing lines, markers, bars, ..." etc.

@story645
Copy link
Copy Markdown
Member

story645 commented Jan 23, 2025

From my point of view people need to be able to make a figure, make and axes, and then make visualizations in those figures and axes. The User Guide as it stands does not have that last step. They are all underscored by objects, so I'm not following what that has to do with explaining how to use the library.

Because those sections are focused on explaining what a figure (object) is and how to create it, what an axes (object) is and how to create it, etc.

@timhoffm is also suggesting by artist here, just implicitly:

But within these sections again, it should not be methods but "drawing lines, markers, bars, ..." etc.

Because each of those visual elements is backed by a different artist.

@timhoffm
Copy link
Copy Markdown
Member

@timhoffm is also suggesting by artist here, just implicitly:

But within these sections again, it should not be methods but "drawing lines, markers, bars, ..." etc.
Because each of those visual elements is backed by a different artist.

To be precise, I'm suggesting by kind of visualization. There's a strong parallel between "kind of visualization" and Artists, but it's not exactly 1:1 (lines = Line2D, markers = Line2D). And my point is that the user guide should be written from the perspective what the user wants (e.g. a color-coded visualization of 2D data) not what we provide (a method imshow() or an AxesImage). We should structure by the former and explain how to realize it using the latter.

@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Jan 23, 2025

And my point is that the user guide should be written from the perspective what the user wants

Sure agreed. That is basically what the current version does. It explains the type of data to be visualized and then explains the different methods to accomplish that. We don't want these to be completely abstract. Users should learn out names for things. And they should also learn the differences between the methods.

@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Jan 23, 2025

Because those sections are focused on explaining what a figure (object) is and how to create it, what an axes (object) is and how to create it, etc.

I think you are making a distinction here that the original author of those sections didn't intend.

However, the practical difference between an Axes object and a Line2d object is that you need to call many methods on the Axes object to use the library at all (or use pyplot). On the other hand, you could make hundreds of complicated visualizations using Line2d artists and not know any of the methods on that artist, or that the artist object even exists.

Not at all to argue that these sections should avoid mentioning or using artists, just that I don't think that should be a focus at the top level.

@story645
Copy link
Copy Markdown
Member

story645 commented Jan 23, 2025

That is basically what the current version does. It explains the type of data to be visualized and then explains the different methods to accomplish that.

But what @timhoffm (and I agree) is saying is it should be grouped by visualization task. I think some of this is fixable with subheadings saying things like "Plotting lines", etc...

I think what's getting jumbled up here is that method and task aren't a 1:1 either. Like scatter plots can be made using .plot or .scatter, line plots can be made using .plot, and for special types of lines .vlines, .hlines, .axline, .event and
.stem, .stair and .step (all of which I think can be accomplished with .plot)

@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Jan 23, 2025

But what @timhoffm (and I agree) is saying is it should be grouped by visualization task. I think some of this is fixable with subheadings saying things like "Plotting lines", etc...

The current headings are:

  • Plotting Methods
    • Pairwise data
    • Statistical plots
    • Gridden plots
    • Unstructured data
    • 3d and volumetric data

The sections start with a short intro about the type of data to be visualized, and then are indeed subsectioned by method type. So taking "Pairwise data" , it has

  • plot
  • fill_between, fill_betweenx, and stackplot
  • errorbar
  • scatter
  • bar and stem
  • step and stairs

we could maybe expand the description of things you do with pairwise data a bit at the beginning, but this is pretty obvious thing to do with your data. We could also change the headings, but to what without being too verbose? "plot" -> "plotting lines and markers, one color and marker at a time" "fill_between"-> "indicating regions around a line"?

The first sentence of each of these sections basically says what the primary goal of the method is, so in my opinion it's clearest just to use the method name. New users may not know our names for things, but the hope is they can realize they want to plot pairwise data, and then skim the plots for the method most suited to their visualization and then read more.

@jklymak jklymak force-pushed the doc-plotting branch 2 times, most recently from 1882990 to 9b56ba2 Compare January 23, 2025 20:20
@tacaswell tacaswell added this to the v3.11.0 milestone Apr 24, 2025
Comment thread galleries/users_explain/plotting/threed.py Outdated
Comment thread galleries/users_explain/plotting/unstructuredgrid.py Outdated
@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Apr 13, 2026

My only request would be that you add a documentation section next to https://matplotlib.org/devdocs/devel/document.html#examples-guidelines and https://matplotlib.org/devdocs/devel/document.html#plot-types-guidelines that describes intent and recommendations of this new part of the docs.

Sure, done. I added a section on tutorials as well since that was missing, and updated the table to have a bit of info about the goal of each section of the galleries, made that a subsection, and added a link near the table to the subsection.

Note this is a separate commit: 2bb91bb

Edit: see https://output.circle-artifacts.com/output/job/f5d8cef0-00ee-4839-9149-67e13aa61731/artifacts/0/doc/build/html/devel/document.html#create-examples-and-tutorials

@jklymak jklymak force-pushed the doc-plotting branch 2 times, most recently from 55485af to 2f70bc2 Compare April 13, 2026 17:19
@timhoffm
Copy link
Copy Markdown
Member

On a side note, I believe our "User Guide" is both "Explanation" (e.g. Introduction to Artists) and "How-to" (e.g. Faster Rendering Using Blitting). That may or may not be good, but is out of scope for this PR.

The important part is that we agree on which category "Plotting data" is. I agree with @jklymak (#29124 (comment)) that "Plotting data" is a "How-to". Essentially,

  1. How do I plot [data type]"? - Decide on the kind of visualization you want and follow up with the respective plotting function in 2.
  2. How do I use [plotting function]? - Learn how to pass your data into the function and some relevant customizations (mostly kwargs).

I like the tutorials section. 👍

I'm unclear on the user guide section, which also stems from the fact that the user guide is currently "How-to" and "Explanation". I would not bother on trying to define this now and here. Just the "plotting data" section should be enough.

@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Apr 13, 2026

I'm unclear on the user guide section, which also stems from the fact that the user guide is currently "How-to" and "Explanation". I would not bother on trying to define this now and here. Just the "plotting data" section should be enough.

Sorry to cross talk in #31496...

As I state there, I think it's awkward to discuss a subsection of the User Guide like it is standalone. The point of this PR is that it is a gap in the current User Guide.

@story645
Copy link
Copy Markdown
Member

story645 commented Apr 13, 2026

The important part is that we agree on which category "Plotting data" is. I agree with @jklymak (#29124 (comment)) that "Plotting data" is a "How-to". Essentially,

How do I plot [data type]"? - Decide on the kind of visualization you want and follow up with the respective plotting function in 2.
How do I use [plotting function]? - Learn how to pass your data into the function and some relevant customizations (mostly kwargs).

The plot type gallery solves one, the links out into the API docs which point to the minigallery that goes to the examples solves two. So what's the value add here? ETA2; What I mean is, what makes this different enough from the minigallery->examples pathway that it's worth maintaining a whole set of docs rather than signposting that pathway better? (We can always add a "what's next? learn to plot through the example gallery" card)

ETA: in particular I'm thinking of examples like pie.

Comment thread doc/devel/document.rst
Comment on lines +975 to +976
- Source location
- Build location
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can these be one column/ two lines?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't take this suggestion..

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It just takes up a lot of white space/is particularly bad on mobile:
Screenshot_20260413-221444.png

Comment thread doc/devel/document.rst Outdated
* - ``examples``
- :file:`galleries/examples`
- :file:`doc/gallery`
- Visual examples that demonstrate features and common (and uncommon) usage patterns.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Visual examples that demonstrate features and common (and uncommon) usage patterns.
Demonstrations of library features and usage patterns

examples is given from the name, not all are visual, and common+uncommon means all

Comment thread doc/devel/document.rst Outdated
Comment on lines +995 to +996
See :ref:`organization-of-examples-and-tutorials` for more information about how to
choose what content goes in which gallery.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like the word choose here b/c these should be well defined enough that it's clear where the docs go. Like we know if something goes into API b/c it's API docs.

Comment thread doc/devel/document.rst Outdated
Comment on lines +1168 to +1169
Outside of the API reference documentation, the Matplotlib documentation consists of
a collection of examples and tutorials.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Outside of the API reference documentation, the Matplotlib documentation consists of
a collection of examples and tutorials.
the Matplotlib usage documentation consists of the following narrative documentation: user guide, tutorials, and examples.

disappearing user guide here does not convey that it's its own thing and not examples/tutorials.

Comment thread doc/devel/document.rst

Plot types guidelines
---------------------
Tutorials are step-by-step guides for learning broader workflows in more depth.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you define broader workflows b/c I'm struggling w/ how artist tutorial is a broader workflow. Granted, I also think it should be folded into the user guide entry on artists.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are general guidelines, not meant to cover everything that already exists or be overly binding.

Comment thread doc/devel/document.rst
Comment on lines +1274 to +1276
They should contain explanation and instruction, and can be more cross-cutting
than individual sections of the user's guide and more in-depth than an
example. Tutorials need not be limited to assuming basic Matplotlib knowledge,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the criteria for more cross-cutting and more in depth?

Comment thread doc/devel/document.rst
material into stages with clear subsections, and make it easy for readers to see
what each section adds and when to refer to other documentation for deeper
detail on a specific feature. A table of contents may be helpful for longer
tutorials.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion but the thumbnail?header image should usually be the finished product as that'll give folks the "why do I want to read this?"

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The purpose here is not to tell people how to write these. Thumbnails are covered above.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rest of this paragraph is content suggestions: sections/headings/toc. This is just another content suggestion.

Comment thread doc/devel/document.rst
Comment on lines +1198 to +1199
the examples. Deep dives into one feature set should usually live elsewhere and
be linked as further reading when appropriate.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where? example gallery?

Comment thread doc/devel/document.rst
Comment on lines +1182 to +1184
be read linearly and build on previously introduced concepts. It should
contain more explanation and instruction than the gallery of examples, while
still being concise and to the point. Cross-link related documentation
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's the sticking point - the difference between user guide and example shouldn't just be that user guide is longer.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"...just be that the user guide is longer" is not what these sentences are meant to convey.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's what "more than" conveys.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, I think the full three paragraphs conveys a lot more than that. If that is all you got out of it, edit suggestions, or comments about what you didn't understand are welcome to make the point clearer.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I'm not understanding is if structurally/modally there's supposed to be anything different about a user guide page vs. a tutorial or user guide page vs. an example.

Like "Customizing Plots" could be a tutorial or a user guide entry or a gallery example. We have it in all three flavors - current rc guide, lifestyle of a plot has a lot of styling guidance, examples w/ a styling:type tag. So what distinguishes it as a user guide entry if it's not just "more explanation"? minimal/focused code is the same guidance as for examples.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • The User Guide should be intentionally organized and largely readable in sequence, at least within each section and ideally across the full guide.
  • Tutorials should teach end-to-end visualization workflows, not isolated feature explainers, and should stand alone for readers who have completed the User Guide.
  • Examples should be quick visual references: easy to scan, easy to copy, and easy to adapt.

Overlap is expected. But instead of asking “Where does page X belong?”, we should ask, “What does the User Guide need in order to be reasonably comprehensive?”

Comment thread doc/devel/document.rst
Comment on lines +1191 to +1193
describes the content of the page. The code examples should be minimal and
focused on the concept being explained. Avoid including extraneous code that
does not directly contribute to the explanation.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly, to me some of the examples in this PR don't follow these guidelines. Which I agree w/ in theory, which makes me wonder if we have different definitions of "concept being explained" and "extraneous".

What I'm thinking of in particular is what I see in the comments as digressions on implementation details (aspect, aliasing, color cycles). And also (as I think Tim also mentioned) most of the focus in the preamble of these functions is on the data - I'm feeling like the plotting concept is getting completely lost here.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Matplotlib is a scientific data visualization tool. Usually scientists have data they want to plot, so I think discussing the data types (pairwise, gridded, etc) and various way to represent that data is a relatively useful way to contextualize. It is also exactly how Plot Types is organized. I'm not sure what "plotting concept(s)" are bing lost.

As for digressions, there may indeed be some. It will be a question whether we should add more, or reduce them, but I expect the former. The ones included were ones I felt were important.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ones included were ones I felt were important.

I think if they're important enough to be mentioned, then they should get a proper subsection/paragraph heading w/ explaination on why you're telling them this. Otherwise it feels like a distraction from whatever they're supposed to be learning in the specific example. Mostly cause someone reading the plotting section of the user guide is likely missing all the context for why this is important to know.

Copy link
Copy Markdown
Member

@story645 story645 Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is also exactly how Plot Types is organized.

B/c plot types is answering the "what chart types can I choose from?" question, while I think user guide is supposed to answer "what can I do w/ the chart type I chose?

I'm not sure what "plotting concept(s)" are bing lost.

For example, the start of the gridded data section is a paragraph and a half on gridded data. There isn't even a brief introduction to the plotting methods, something like "the regular gridded data plotting methods all plot variants of a heatmap, which is a visualization where the data values $Z_{i, j}$ are encoded as colors. Matplotlib does this encoding by mapping the data to color look up values using normalization and then using the normalized values to look up values in the colormap".

Some of this is addressed on the specific methods, but like we both know that it's the $norm \circ cmap$ pipeline that messes up newbies to mpl and that's the core of the heatmap/imshow visualizations,
so that should be top level. Especially since it applies to all the gridded methods. It's definitely not less important than aspect, which somehow gets more emphasis - which also you introduce vmin/vmax before norm but vmin/vmax is just implicit norm.
And the reason for telling users what a heatmap is even though we expect they know is to ensure we're using the same terms.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, first sentence fixed, but note this section is not just about colormapping as contours need not be colored.

As for mapping using norms, a full explanation is linked, and maybe more could be explained here. However, I consider that somewhat intermediate, and I bet 95% of our users never create a norm directly.

Copy link
Copy Markdown
Member

@story645 story645 Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but note this section is not just about colormapping as contours need not be colored

That's why I (and I think Tim) were pushing for some/sub organization under viz type. Group heatmap like functions together so all the shared concepts (colormaps, norms) can be discussed under that header and then the function how tos can be streamlined to the function specific differences. Main reason plot types gallery doesn't have that sub organization is b/c we don't go into that level of detail there. Bonus is easier to edit when we add explainations about colorizer later.

However, I consider that somewhat intermediate, and I bet 95% of our users never create a norm directly.

I've probably been using Matplotlib almost 20 years and I don't think I've ever really cared about the aspect but you think it's very important for understanding how to use imshow. Which fine, I think norm is one of those things that's really important to understand even if you don't make one directly b/c it underpins all the colormapping (at least currently).

@timhoffm
Copy link
Copy Markdown
Member

So what's the value add here? ETA2; What I mean is, what makes this different enough from the minigallery->examples pathway that it's worth maintaining a whole set of docs rather than signposting that pathway better? (We can always add a "what's next? learn to plot through the example gallery" card)

ETA: in particular I'm thinking of examples like pie.

Examples are rather anecdotal. They teach by demonstration, rather than by explanation or instruction. Because the visualization and preview image have a strong weight, I'd consider it rather an anti-pattern for examples to have many plots.

In that sense, some of the examples, including pie are actually very much the type of content, I'd like to see under "Plotting data" and should be moved there. (they are currently in the examples category because we don't have a dedicated place for them).

Another example of "Plotting data" content: plot() should roughly describe accepted data (plot(y), plot(x, y), plot(x, Y)), styling using kwargs, styling shortcuts using fmt. That's not something we can reasonably put into an example.

@story645
Copy link
Copy Markdown
Member

story645 commented Apr 13, 2026

some of the examples, including pie are actually very much the type of content, I'd like to see under "Plotting data" and should be moved there

Agree, argued something similar in #27459 or the discussions around it, (which honestly my motivation there is the same as one of my concerns here - focus is too much on what's being plotted, not enough on how to plot it) and discussions around it. But that's also a very different approach from the one taken in this PR. I'm also deeply skeptical of the "well then rework on next PR to that style" b/c it will run headlong into the same issues as #27459

@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Apr 13, 2026

some of the examples, including pie are actually very much the type of content, I'd like to see under "Plotting data" and should be moved there.

Yes, that'd be great - not including pie is an oversight, not a deliberate omission. There are likely others.

I'd indeed go through the Examples and do some pruning - some of the material is already cribbed from examples, and probably more could be done.

@timhoffm
Copy link
Copy Markdown
Member

As a side note/feedback: I wasn't able to easily see whether #29124 (comment) and the subsequent force-push did change pie or whether that was just a communication of intent. This unerpins my point that the PR is too large in scope and content to be reviewable. While I rejoined the conceptual high-level discussion, I still won't systematically look at the actual content of the PR.

@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Apr 15, 2026

No pie is not included. As I'm sure many other things are not either.

@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Apr 15, 2026

A bit of backstory.

The documentation has had some long-recognized structural challenges:

  • The galleries were extensive, but not always easy to navigate consistently.
  • The Tutorials included strong content, but the Beginner/Intermediate/Advanced split was not always the most effective way to support topic-focused learning.
  • Tutorial pages also served different purposes: some were walkthroughs, while others were closer to reference material.
  • The User Guide contained important material, but over time it also became a home for a wide range of topics that did not clearly fit elsewhere.

Addressing this required clearing a few technical hurdles:

With those pieces in place, we could restructure the User Guide as a sphinx-gallery and move substantial tutorial material into it (#25395), while also adding new content to fill gaps.

Alongside technical constraints, there was also a process challenge: many good ideas were discussed over time (summits, surveys, working groups, sprints, outside expertise), but progress remained slow. Those approaches can still be valuable, but for this specific problem it also seemed important to make practical, iterative improvements in parallel.

My view is that this work was less about finding a perfect information architecture up front and more about putting a clear starting structure in place, then improving it through feedback. Once that structure exists, feedback tends to become more specific and actionable.

From that perspective, I believe #29124 is a constructive step forward, even if it is not final. If the project prefers a different direction, it can choose that. Either way, I hope this helps move the docs discussion from general complaints to concrete improvements.

@story645
Copy link
Copy Markdown
Member

story645 commented Apr 15, 2026

but over time it also became a home for a wide range of topics that did not clearly fit elsewhere
Once that structure exists, feedback tends to become more specific and actionable.

Most of the discussion here and in #31496 is about getting consensus on structure and content guidelines, so I don't think anyone disagrees on the problems.

The reason for the discussion is I think:

  • agreement on what scope to try and avoid scope creep
  • agreement on structure b/c structure is really difficult to change, especially b/c everything will need redirects
  • enough agreement on scope/structure/mode that folks understand what a new addition to this section should look like.

A major reason the user guide doesn't get touched is it feels like there really isn't space to make new changes - nobody knows what "improving" looks like. Which that's an underlying goal here, to get some clear consensus on what "improving" means b/c we can translate that into guidance and actionable tasks.

ETA: otherwise I think we're mostly just kicking this discussion down the road 'til the next time someone who feels empowered to make these changes makes them, and our goal should be to empower any contributor to make them.

@story645
Copy link
Copy Markdown
Member

story645 commented Apr 17, 2026

No pie is not included. As I'm sure many other things are not either.

So let me be a bit more concrete here on one of the main things I (and I think @timhoffm) am looking for: how does pie get added?

Can't wholesale move pie features b/c the current structure puts all plots for each data type on one page. Can't c&p the contents into the current doc b/c that'd take up the whole page & the structure doesn't meld w/ what's there. Removing the structure kills what makes the example useful - easy to see what each unique parameter does.

Do folks do something else? What? Which parts of pie get highlighted? Or do follow ups restructure to each plot getting its own page? Under broader data type sections? W/ consistent structure/format to follow?

ETA: and just to be explicit/clear, I absolutely do not want you to just add pie to this PR. What I wanna know is what's the path forward for anybody else who'd like to build on this PR.

@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Apr 17, 2026

how does pie get added?

I'd put in statistical plots. I'd have a short explanation of the type of data that gets input, and an example of basic use, followed by a couple of examples of important styling or common advanced use (in this case perhaps colouring the slices, and probably labelling them), and then list and reference the API page and some of the more advanced features in the examples (hatching, exploding, etc).

The goal is that people have a basic idea of what pie is, and what its flexibility is, and then they can drill down. "A couple of examples" would be a judgement call, as would putting it in "statistical plot", as would the level of explanation or how it is explained.

I think this is largely covered by

The material should stay relatively balanced and move between basic and more
advanced concepts in as natural a manner as possible. The goal is for a user to
read the material without having to spend too much effort trying to understand
the examples. Deep dives into one feature set should usually live elsewhere and
be linked as further reading when appropriate.

@story645
Copy link
Copy Markdown
Member

story645 commented Apr 17, 2026

some of the more advanced features in the examples (hatching, exploding, etc).

hatching is basic accessibility, which is not me trying to be annoying so much as trying to flag that "basic" and "advanced" are subjective criteria, usually determined by the conventions of a specific field. Which is how doc prs get derailed on "does this belong?".

That's a big reason why I think "comprehensive" (like pie features) makes more sense for a guide - it's basically a checklist of "is using this parameter explained with an example?"

ETA2: Which this isn't even a request for you to make these changes here, just the direction I think changes should move to improve upon what's here.

ETA: A big goal being that we can then refine the example gallery to be much more focused on fully exploring parameter space. So user guide shows how to use a colormap, example gallery shows all the common and weird ways colormaps can be used.

ETA3: Annotation Guide functionally is this - the basic section and first half of advanced section is basically a comprehensive guide to the annotate method. Also subplot mosaic for the subplot_mosaic method.

@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Apr 17, 2026

hatching is basic accessibility, which is not me trying to be annoying so much as trying to flag that "basic" and "advanced" are subjective criteria, usually determined by the conventions of a specific field. Which is how doc prs get derailed on "does this belong?".

Yes, and that is absolutely why I'm not breaking this PR into 20 or 30 separate PRs, because I have zero interest in having all those debates that end up 10 times longer than the PR.

@story645
Copy link
Copy Markdown
Member

story645 commented Apr 17, 2026

because I have zero interest in having all those debates that end up 10 times longer than the PR.

That's also why this PR isn't moving forward - b/c I don't want us stuck in a situation where every single follow up to this PR ends up in those debates such that nobody makes follow ups.

ETA: And why I think the criteria should be the more objective "comprehensive" rather than the subjective "basic"

@story645
Copy link
Copy Markdown
Member

story645 commented Apr 17, 2026

Also, gonna be super frank here @jklymak. This PR is too big for most folks to sensibly review (1) and you've said you don't have the time (and indicated the inclination) to make significant changes (2). Were this PR authored by almost anybody else, it would at best have been met with a "thanks, we're gonna put this in draft until you have time to really engage with the feedback" if not tagged w/ an autoclose label. It's basically only out of respect for your work and for Tom thinking this PR is a good idea that I (and maybe Tim's motivation is similar) am trying to figure out a consensus for the next steps that will definitely need to happen on account of 1 & 2.

And no "merge despite all my misgivings such that Jody is the only person who can maintain this" is not a path forward.

@story645
Copy link
Copy Markdown
Member

story645 commented Apr 17, 2026

So to be concrete, what I see as maintenance issues here is roughly the same as why it's difficult to review. There's too much content crammed together w/ unclear criteria for inclusion of examples. I'm proposing one plot type per page b/c that keeps each page smallish and well scoped (and therefore easier to review and update) and comprehensive as the inclusion criteria b/c it's fairly objective. You're welcome to disagree, but just either explain why your current PR is more maintainable than my proposal or propose a scheme you think is more maintainable. (Edit, which really, I'd break this PR into 6 - one per plot type section + one for guidelines - not 20 or 30).

@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Apr 17, 2026

I am not sure what you mean by "plot type". I've not seen where you are "proposing one plot type per page" in any detail for me to compare against.

Guessing: If you mean one plot method per page (eg plot, scatter etc), that would be unreadable as a table of contents, even if you ordered them somehow, which I hope is not what you are talking about.

Barring that, organizing the individual methods is basically what this PR does, but instead of one page per method, it has one page per visualization type, and the individual methods are subsection. I would argue that isn't "cramming" anything, or any more difficult to maintain than breaking up into individual files. It also has the benefit of having an introduction explaining the grouping. If you want the listing of all the individual plotting methods, just increase the index depth.

With regards to comprehensiveness, that would basically mean taking 95% of the example material and moving it into the User Guide. That maybe is something that could be considered, but I think "get a good idea and link out" is easier on the reader and a good start.

@story645
Copy link
Copy Markdown
Member

story645 commented Apr 17, 2026

Guessing: If you mean one plot method per page (eg plot, scatter etc), that would be unreadable as a table of contents, even if you ordered them somehow, which I hope is not what you are talking about.

This is the proposal from #31496 (comment) of repurposing the plot gallery content pages as plot method guides. So essentially the plot type gallery becomes the TOC and on the user guide we can just have the first level plot type that you have here.

ETA: Which, it's quite likely our users would just jump to the plotting guide and then go back to the other sections of the user guide as needed b/c plotting is the priority.

It also has the benefit of having an introduction explaining the grouping.

This would move to an index page for each of the folders that already exist in plot_types, similar to what we already do for text and colors

I would argue that isn't "cramming" anything, or any more difficult to maintain than breaking up into individual files.

It's a lot harder to find/keep place in the doc b/c it has so much going on. That's what's making it so hard to review. And to see if/what is missing. Narrowing to one method per page makes it easier to (relatively quickly) get a per method overview w/o having to think about the other things on that page.

that would basically mean taking 95% of the example material and moving it into the User Guide.

This is why I'm so adamant on one example per kwarg visual overview of API (what the parameter does vs all the ways to use the parameter). For pie it would be pie features, not everything in pie and polar examples. For histogram it would mostly just be histogram bins, density, and weight - whatever my objections to how it's structured/where it's focused, it's an overview of the parameters/using the function. Which right, this belongs in user guide but would absolutely swamp the stats section.

@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Apr 17, 2026

I think your proposal, as described here, would be useful reference material, but would be a level below (or adjacent) to a User Guide.

Narrowing to one method per page makes it easier to (relatively quickly) get a per method overview w/o having to think about the other things on that page.

Separate pages are hard to read together. You need to read your section, go back, and then click the next subject. That is fine for reference material - it breaks up a narrative.

This is why I'm so adamant on one example per kwarg

For most methods this would be unreasonable level of detail for a beginning User Guide; I'd argue a comprehensive kwarg scan would be better placed in a "Further reading section" and live elsewhere.

Overall, what you are suggesting sounds like reference material to me, not a Users Guide. I don't object to that at all, but it is orthogonal to the proposal here.

(BTW I did not see that scoped out in the comment you linked, which is by @timhoffm #31496 (comment) and definitely does not say that there should either be a single page per method nor that there should be a kwarg scan on each page, so perhaps you linked the wrong comment?)

@timhoffm
Copy link
Copy Markdown
Member

Hm, I may slowly better understand the different motives here. @jklymak I now wrote a documentation section, which I requested for this PR here to the scope I now believe you are trying to deliver. Is this correct?

Plotting data

The section Plotting data of the Users guide showcases plot types as a guided walk-through. The user should learn what is available (e.g. pairwise data can be visualized as lines and/or markers) and how to achieve this (e.g. using plot()). Focus is primarily on the visual outcome. We explain plotting functions and relevant parameters only to the level necessary to generate that outcome (e.g. do not discuss all call signatures for the data values of plot(); do mention the marker parameter, but do not extensively discuss styling like color or linewidth).

Show by visual example. The user should have a basic first idea by only looking the plots. Each visualization should be introduced by a brief explanation to set the stage. Cross-link related documentation; primarily the API documentation, but also tutorials and examples if appropriate.

That at least would make sense to me, and in this context, putting multiple functions on one page is fine. There are some places the PR deviates a bit from the scope and could be more concise, but overall this is a direction we can go.


IMHO @story645 and me are hitting another scope. @story645 please correct if I'm wrong.

Plotting functions

The section Plotting functions of the Users guide explains common usage patterns and parameters of common plotting functions. The user should learn how the respective function is typically used. Explain typical call signatures (e.g. for plot(): x, y, y, x, Y, but not x, y, x2, y2) and parameters (e.g. for plot()` the fmt parameter, common styling). Each mentioned aspect should be illustrated in a plot. Use separate plots or aggregate multiple aspects (e.g. common styling parameters) into one plot as appropriate.

The section should contain one page per function. Link that page from Plot types and from the function's API. Cross-link related documentation; primarily the API documentation, but also tutorials and examples if appropriate.

I think we can do both and with good scoping we reduce the risk of creating a mess. Feel free to comment on the overall idea of this scope distinction and/or specific definitions I wrote in these doc proposals (preferably first general idea, and later bikeshedding specifics).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Documentation: devdocs files in doc/devel Documentation: examples files in galleries/examples Documentation: plot types files in galleries/plot_types Documentation: user guide files in galleries/users_explain or doc/users status: needs comment/discussion needs consensus on next step

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants