This is a rather annoying speed bump when working with multiple files as you have to renavigate on every task change.
Export and Save As dialogues are both effected.
I'd argue the current method doesn't even need to be preserved as the only time you would really prefer last used directory is for unsaved files which can probably be easily checked.
I like what notepad++ does for unsaved files, it will default to the directory of the last saved tab that had focus.
So if your working on /foo/file1.md and jump to an unsaved file if you try to save it will open a save dialogue under /foo/