Syncing Modals with the Back Button — Designing a Reusable useHistoryBack Hook

Opening — Back Button Leaves the Page Instead of Closing the Modal

On mobile, pressing back while a modal is open should close the modal first, obviously. But my initial implementation ignored the modal and navigated away from the page entirely.

UX-wise, that's brutal. Opening an info modal and closing it with the back button is a natural move — and mine made it exit the site.

There were several similar modals in this project — NavMenu, InfoModal, SearchPalette, Harmony recording guide, Mars image viewer, and more. Implementing each separately would scatter the logic, so I consolidated into one shared hook.

First Attempt — Add an Entry with pushState

The intuitive approach. Push a history entry when the modal opens; when the back pops it, close the modal.

// first attempt
useEffect(() => {
  if (!isOpen) return;
  history.pushState({ modalOpen: true }, '');

  const onPop = () => setIsOpen(false);
  window.addEventListener('popstate', onPop);
  return () => window.removeEventListener('popstate', onPop);
}, [isOpen]);

The Trap I Hit

In a Next.js App Router environment, manual pushState entries "swallow" popstate on back.

For example:

1. User on /galaga
2. Opens modal → pushState adds entry
3. User hits back → popstate fires → modal closes ✓
4. User hits back again → nothing happens

Manually-inserted entries and Next.js router-managed entries tangle, producing unexpected state.

The Fix — The replaceState Marker Pattern

Instead of pushState, just stamp a flag onto the current entry:

history.replaceState({ ...history.state, modalOpen: true }, '', location.href);

• No new entry added → no interference with router history
• Adds modalOpen: true as a marker to the current state object
• Pressing back still goes to the real previous page (correct)
• But the fact that "the current page's state had modalOpen" is preserved

Modal-close UX goes through a separate channel. The OS back button works normally (goes back a page), while modal close is routed through other events only — X button, backdrop click, ESC.

If you still want "back button closes the modal", you need a different approach.

Final Design — dismiss Return + popstate Branch

The pattern I actually shipped. Two separate close paths:

1. "Close via back button" — call history.back() → popstate fires → modal closes
2. "Close via X / ESC / backdrop" — close modal only, don't touch history

export function useHistoryBack(isOpen: boolean, onClose: () => void, marker: string) {
  // preserve state, just stamp the marker (not pushState)
  useEffect(() => {
    if (!isOpen) return;
    history.replaceState({ ...history.state, [marker]: true }, '', location.href);

    const onPop = () => {
      // if popstate fires and our marker is gone, close the modal
      if (!history.state?.[marker]) {
        onClose();
      }
    };
    window.addEventListener('popstate', onPop);
    return () => window.removeEventListener('popstate', onPop);
  }, [isOpen, onClose, marker]);

  // function the user calls when explicitly closing
  const dismiss = useCallback(() => {
    if (history.state?.[marker]) {
      history.back(); // simulate back button
    } else {
      onClose(); // direct close
    }
  }, [onClose, marker]);

  return dismiss;
}

Usage

function Modal({ isOpen, onClose }: Props) {
  const dismiss = useHistoryBack(isOpen, onClose, 'myModalOpen');

  return (
    <div onClick={dismiss}>
      {' '}
      {/* backdrop click */}
      <button onClick={dismiss}>✕</button> {/* X button */}
      {/* ... */}
    </div>
  );
}

• Backdrop / X / ESC → dismiss() → history.back() → popstate → modal closes
• OS back button → popstate fires directly → modal closes
• Either way passes through the same path

The formula "one browser-back = one modal closed" stays consistent.

Other Traps

Never Call history.back() in cleanup

// ❌ Dangerous
useEffect(() => {
  return () => history.back(); // back on unmount?
}, []);

If this fires during Next.js soft navigation, it can cancel the routing. history.back() should only come from explicit user actions.

Watch Out for Nested Modals

If a modal opens another modal inside, use different marker names so they don't interfere.

useHistoryBack(isOpen, onClose, 'outerModalOpen');
// ...
useHistoryBack(innerOpen, innerClose, 'innerModalOpen'); // different marker

On popstate, each hook independently checks which marker is in history.state.

Pre-injecting the Marker Skips pushState Ordering Bugs

Stamping the marker via replaceState right before opening keeps the order with existing history entries clean. This pattern is why it coexists with the Next.js router without conflict.

Where It's Used

Places in this project now using useHistoryBack:

• NavMenu (hamburger)
• InfoModal (page info)
• SearchPalette (⌘K)
• Constellation info panel
• Harmony recording guide
• Mars image detail viewer
• ThreeBody help
• Modals inside Devlog /devlog/[slug]

Eight places, same pattern. Each component needs one hook call.

Retrospective

When you first implement a modal, a single isOpen state feels sufficient. But once you factor in mobile UX and browser history sync, the design turns out to need real care.

The key is maintaining the correspondence "close modal = one back press". Break that, and users start wondering "how many times do I need to press back to actually leave this page?"

The replaceState + dismiss pattern keeps that formula intact while staying compatible with the router. If your SPA has many modals, it's a hook worth packing up and reusing.