Explanation

Pygame's window

What you've already noticed is that a KEYUP event is sent whenever pygame's window loses focus. The reason for this is that pygame's key events are, for the most part, wrappers of SDL's key events (SDL is a library written in C with low level access to many different components, one of which is graphics hardware) and if you look at SDL_KeyboardEvent's data fields, you can see that one data field is called windowID:

The windowID data field is responsible for holding window ID of the window from which it's grabbing keyboard inputs - only while the window is in focus as otherwise the window has no information over the keyboard inputs. As a protection measure, SDL's window automatically sends an artificial KEYUP event whenever the window specified in windowID loses focus (which in turn makes pygame send the KEYUP event as well). Another thing to note is that the OS sends multiple KEYDOWN events to a window whenever a key is held, but SDL automatically ignores every KEYDOWN event other than the first one.

Tkinter's window

Tkinter's window, as any other window, also gets keyboard inputs from the OS - but raw inputs. That's why, when you hold a key for a while, tkinter's window shows all the KeyPress events it gets from the OS. When a tkinter's window loses focus, it doesn't send any KeyRelease event as it hasn't gotten any from the OS (unlike SDL's window, where the KEYUP event is generated artificially).

Pygame inside of Tkinter's window

When pygame uses tkinter's window, it can't catch the OS's keyboard inputs - only the keyboard events from tkinter. The reason for this is the following line of code:

os.environ['SDL_WINDOWID'] = str(embed_frame.winfo_id())

SDL_KeyboardEvent now uses embed_frame's windowID. That means that all the keyboard events pygame catches will be from tkinter. This is why pygame inside of tkinter doesn't have an extra KEYUP event when the tkinter's window loses focus, where as it has the KEYUP event when it uses it's own window.

Solution

Unless you're willing to edit and compile pygame, tkinter or SDL, there is no way to solve this using pygame's default event queue. But you can still use a custom event handler or write your own and work around this issue.

For this example, just having a key listener is enough (solution uses pynput):

import tkinter as tkimport pygameimport osfrom tkinter.simpledialog import askstringfrom pynput import keyboardon_enter = Falsedef on_press(key):    # Uses global 'on_enter' variable    global on_enter    # If key's name is a single letter ('a', '1', etc.) then 'key.char' is used,    # otherwise ('shift', 'enter', etc.) 'key.name' is used    try:        key_ = key.char    except AttributeError:        key_ = key.name    # When 'shift' is down - set color to dark* green    if key_ == 'shift':        screen.fill((50, 205, 50))    # When 'enter' is down - set color to white, run 'askstring'    elif key_ == 'enter':        screen.fill((255, 255, 255))        on_enter = Truedef on_release(key):    # If key's name is a single letter ('a', '1', etc.) then 'key.char' is used,    # otherwise ('shift', 'enter', etc.) 'key.name' is used    try:        key_ = key.char    except AttributeError:        key_ = key.name    # When 'shift' is up - set color to white    if key_ == 'shift':        screen.fill((255, 255, 255))root = tk.Tk()root.geometry("200x100")embedding_pygame_and_showing_the_feature = Trueif embedding_pygame_and_showing_the_feature:    embed_frame = tk.Frame(root)    embed_frame.pack(fill='both', expand=True)    os.environ['SDL_WINDOWID'] = str(embed_frame.winfo_id())    os.environ['SDL_VIDEODRIVER'] = 'windib'pygame.init()screen = pygame.display.set_mode((200, 100))screen.fill((255, 255, 255))# Runs 'on_press' when it detects key down,# runs 'on_release' when it detects key uplistener = keyboard.Listener(on_press=on_press, on_release=on_release)listener.start()while True:    # on_enter == True only when 'enter' is down    if on_enter:        # This display flip is necessary as keyboard.Listener runs        # in a separate thread - 'on_enter' can change at any point        # in the while-loop        pygame.display.flip()        askstring(' ', ' ', parent=root)        on_enter = False    root.update()    pygame.display.flip()