Fix a bunch of temp full-screen glitches

Steps to reproduce fixed glitches were:
* Change any editor to be file browser from menu, Ctrl+O *from the file browser area*, Esc -> area reset to what it was before changing to file browser initially
* Ctrl+O from any area, F12, Esc -> returns to initial editor in full-screen (expected is file browser in full-screen)

Fixes T46229

Core of the fix is removing old area from spacedata list when going back to previous area (see ED_area_prevspace -> BKE_spacedata_remove). Also, when creating a new temp area we now don't exit old area anymore (needed so SpaceFile->op is kept, but it also makes sense in general)
Aaand finally removes some ugly hacks.

Tested quite a bit, so I think it's safe to apply (besides of remark below), just would like to get things double checked and confirmed. After all, this full-screen stuff finally starts to feel like it's working :P

Note, there's still a memory leak when quitting Blender with temp area open. Haven't found out how to solve yet, but it's not that important for review anyway.

Reviewers: campbellbarton, brecht

Reviewed By: brecht

Subscribers: plyczkowski, Blendify

Maniphest Tasks: T46229

Differential Revision: https://developer.blender.org/D1531

2 files changed, 8 insertions, 7 deletions

diff --git a/source/blender/windowmanager/intern/wm_event_system.c b/source/blender/windowmanager/intern/wm_event_system.c
index 4239ccc1b69..24d4144ec17 100644
--- a/source/blender/windowmanager/intern/wm_event_system.c
+++ b/source/blender/windowmanager/intern/wm_event_system.c
@@ -1792,9 +1792,12 @@ static int wm_handler_fileselect_do(bContext *C, ListBase *handlers, wmEventHand
 				if (sa->prev) {
 					sa = sa->prev;
 				}
-				ED_area_newspace(C, sa, SPACE_FILE);     /* 'sa' is modified in-place */
+				ED_area_newspace(C, sa, SPACE_FILE, true);     /* 'sa' is modified in-place */
 				/* we already had a fullscreen here -> mark new space as a stacked fullscreen */
-				sa->flag |= AREA_FLAG_STACKED_FULLSCREEN;
+				sa->flag |= (AREA_FLAG_STACKED_FULLSCREEN | AREA_FLAG_TEMP_TYPE);
+			}
+			else if (sa->spacetype == SPACE_FILE) {
+				sa = ED_screen_state_toggle(C, CTX_wm_window(C), sa, SCREENMAXIMIZED);
 			}
 			else {
 				sa = ED_screen_full_newspace(C, sa, SPACE_FILE);    /* sets context */
@@ -1823,11 +1826,9 @@ static int wm_handler_fileselect_do(bContext *C, ListBase *handlers, wmEventHand
 
 			if (val != EVT_FILESELECT_EXTERNAL_CANCEL) {
 				ScrArea *sa = CTX_wm_area(C);
-				const SpaceLink *sl = sa->spacedata.first;
-				const bool was_prev_temp = (sl->next && sl->next->spacetype == SPACE_IMAGE);
 
 				if (sa->full) {
-					ED_screen_full_prevspace(C, sa, was_prev_temp);
+					ED_screen_full_prevspace(C, sa);
 				}
 				/* user may have left fullscreen */
 				else {
diff --git a/source/blender/windowmanager/intern/wm_window.c b/source/blender/windowmanager/intern/wm_window.c
index 932f9994a4a..688be21cdd0 100644
--- a/source/blender/windowmanager/intern/wm_window.c
+++ b/source/blender/windowmanager/intern/wm_window.c
@@ -678,10 +678,10 @@ wmWindow *WM_window_open_temp(bContext *C, const rcti *rect_init, int type)
 	CTX_wm_area_set(C, sa);
 	
 	if (type == WM_WINDOW_RENDER) {
-		ED_area_newspace(C, sa, SPACE_IMAGE);
+		ED_area_newspace(C, sa, SPACE_IMAGE, false);
 	}
 	else {
-		ED_area_newspace(C, sa, SPACE_USERPREF);
+		ED_area_newspace(C, sa, SPACE_USERPREF, false);
 	}
 	
 	ED_screen_set(C, win->screen);