I'm James Hague, a recovering programmer who has been designing video games since the 1980s. This is Why You Spent All that Time Learning to Program and Organizational Skills Beat Algorithmic Wizardry are good places to start.
Where are the comments?
How to Think Like a Pioneer
Here's an experiment to try at home: do a Google image search for "integrated development environment." Take some time to go through the first several pages of pictures.
Even if you had no idea what an IDE was, the patterns are obvious. There's a big area with text in it, and on the left side of the screen is a pane that looks a lot like Windows Explorer: a collection of folders and files in a tree view, each with icons. Some folders are closed and have little plus signs next to them. Others are expanded and you can see the files contained within them. To be perfectly fair, this project area is sometimes on the right side of the window instead of the left. Variety is the spice of life and all that.
Why have IDEs settled into this pattern of having a project view take up twenty percent or more of the entire left or right side of the window?
The answer is shallower than you may expect. Someone who decides to create an IDE uses the existing programs he or she is familiar with as models. In other words, "because that's how it's supposed to be."
There's no solid reason the project view pane has to be the way it usually is. In fact, there are some good arguments against doing it that way. First, you are typically either looking at the project view or you're editing a file, not doing both at the same time. Yet the project view is always there, taking up screen real estate, sometimes a lot of screen real estate if highly nested folders are expanded. That view is made even wider by having icons to the left of each filename, even though most projects consist of one or two file types and they could be differentiated with color or an outline instead of a contrived icon attempting to evoke "file of Java code."
Second, a long, thin pane containing files and folders doesn't give a particularly deep or even interesting view of a large project. Open a folder and it might fill the whole pane, and all the other folders are now offscreen.
Are there better options? Sure! The first problem, of losing screen space to a persistent view of the project hierarchy, can be alleviated by a hot key that brings up a full-screen overlay. When you want to see the project as a whole, hit a key. Press escape to make it go away (or double-click a file to edit it).
The data presented in this overlay doesn't need to be a tree view. A simple option is to group related files into colored boxes, much like folders, except you can see the contents the whole time. With the narrow, vertical format out of the way, there can be multiple columns of boxes filling the space. Now you can see 100+ files at a time instead of a dozen.
It might make more sense to display modules as shapes, each connected by lines to the modules which are imported. Or provide multiple visualizations of the project, each for a different purpose.
Someone, sometime, and probably not all that long ago, came up with the canonical "project view on the left side of the window" design for IDEs. And you may wonder how that person arrived at that solution. After all, there were no prior IDEs to use for guidance. I think the answer is a basic one: because it's a solution that worked and was better than what came before. No magic. No in-depth comparison of a dozen possibilities. Clearly a way to see all the files in a project is better than a raw window in a text editor where there's no concept of "project" whatsoever. That solution wasn't the best solution that could ever exist in the entire future history of IDEs, but it sure fooled people into thinking it was.
If you want to think like a pioneer, focus on the problem you're trying to solve. The actual problem. Don't jump directly to what everyone else is doing and then rephrase your problem in terms of that solution. In the IDE case, the problem is "How can I present a visual overview of a project," not "How can I write a tree viewer like in all the other IDEs I've ever seen?"