How to use the Playwright library in a Jupyter notebook instead of using a regular .py script (on Wi

I want to use an automated browser and execute my steps with jupyter notebook cells instead of using .py scripts. This works fine with the browser automation library called selenium.

It does not work fine with the library called Playwright. In fact it doesn't work at all. I tried every single line of code they provided in their manual. NOTHING works in jupyter notebooks. EVERYTHING works fine on my machine as long as copy-pasting the same code in some .py file and executing it. Various examples that I'm talking about can be found here: https://playwright.dev/python/docs/intro

I really don't get why I'm unable to make it work in a jupyter notebook, especially if it works fine in literally every .py file.

Edit: Apparently it works on mac but I use windows

4 Answers

The code below works on both MacOS and Linux.

as mentioned in https://github.com/microsoft/playwright-python/issues/480

Jupyter notebook uses asyncio event loop, so you should use async api.

from playwright.async_api import async_playwright playwright = await async_playwright().start() browser = await playwright.chromium.launch(headless = False) page = await browser.new_page() await page.goto("http://whatsmyuseragent.org/") # await page.screenshot(path="example.png") # await browser.close() # await playwright.stop() 

If you use sync API, it will throw an Error like this:

from playwright.sync_api import sync_playwright playwright = sync_playwright().start() ''' Error: It looks like you are using Playwright Sync API inside the asyncio loop. Please use the Async API instead. ''' 

Since Colab notebooks are hosted Jupyter Notebooks, I recommend the following solution for running playwright in your hosted Jupyter instance.

I have only tested in my Google Colab notebook and have not tested in a locally hosted Jupyter instance.

Playwright in Google Colab Solution

If you're experiencing issues running Playwright inside of a Jupyter notebook on Windows, try disabling the event loop policy of the kernel.

if sys.platform.startswith("win") and sys.version_info >= (3, 8): import asyncio try: from asyncio import WindowsProactorEventLoopPolicy, WindowsSelectorEventLoopPolicy except ImportError: pass # not affected else: if type(asyncio.get_event_loop_policy()) is WindowsProactorEventLoopPolicy: # WindowsProactorEventLoopPolicy is not compatible with tornado 6 # fallback to the pre-3.8 default of Selector # asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy()) pass 

If you can't use async API in jupyter notebook, you can try to create a virtual environment for playwright:

In terminal:

# create a virtual environment for playwright python3 -m venv playwright_new source ~/playwright_new/bin/activate pip install playwright ipykernel requests playwright install 

Then, create a kernel link for jupyter notebook:

source ~/playwright_new/bin/activate # create kernel link for jupyter notebook python -m ipykernel install --user --name playwright_new --display-name "playwright_new" # in mac ls /Users/xxx/Library/Jupyter/kernels/ tree /Users/xxx/Library/Jupyter/kernels/playwright_new /Users/xxx/Library/Jupyter/kernels/playwright_new ├── kernel.json ├── logo-32x32.png └── logo-64x64.png # or in linux tree /root/.local/share/jupyter/kernels 

Then, run the python code again.

from playwright.async_api import async_playwright playwright = await async_playwright().start() browser = await playwright.chromium.launch(headless = False) page = await browser.new_page() await page.goto("http://whatsmyuseragent.org/") 

Error debug

 if exec python code throws an Error: Error: Executable doesn't exist at /Users/xxxx/Library/Caches/ms-playwright/chromium-1000/chrome-mac/Chromium.app/Contents/MacOS/Chromium ╔═════════════════════════════════════════════════════════════════════════╗ ║ Looks like Playwright Test or Playwright was just installed or updated. ║ ║ Please run the following command to download new browsers: ║ ║ ║ ║ playwright install ║ ║ ║ ║ <3 Playwright Team ║ ╚═════════════════════════════════════════════════════════════════════════╝ 

In terminal:

# you already install playwright playwright install cd /Users/xxxx/Library/Caches/ms-playwright ls chromium-978106/ ffmpeg-1007/ firefox-1319/ webkit-1616/ # but the folder ms-playwright/chromium-1000 NOT EXISTS # COPY the exists chromium folder with a new name `chromium-1000` cp -r chromium-978106 chromium-1000 

ncG1vNJzZmirpJawrLvVnqmfpJ%2Bse6S7zGiorp2jqbawutJobmpqYm2EdX6OoaawZaSkera%2FxGaroZ1dpbmixdaroKCgpGK5qq7RmqmyZZmjeqJ5ya6nsqyVp3qvu9Oemainm2K2r7%2FTnpidZZ%2Bbera%2FyKeeZpldp7Kowcs%3D