aider: Retrieve the user agent from the browser context after creating a new page in the scrape_with_playwright function.

# Aider chat conversation:

USER: determine the user_agent

ASSISTANT: To determine the `user_agent` in the `scrape_with_playwright` function, we can use the `browser.new_context()` method which allows us to specify a `user_agent` string. We will set a common user agent string for a desktop browser.

Here is the diff for the changes:

```diff
--- aider/scrape.py
+++ aider/scrape.py
@@ ... @@
 def scrape_with_playwright(url):
     with sync_playwright()
^C KeyboardInterrupt

USER: I don't want to SET it. I want to determine what it is

ASSISTANT: To determine the `user_agent` being used by the browser in the `scrape_with_playwright` function, we can retrieve it from the browser context after creating a new page. Here is the diff for the changes:

```diff
--- aider/scrape.py
+++ aider/scrape.py
@@ ... @@
 def scrape_with_playwright(url):
     with sync_playwright() as p:
         try:
             browser = p.chromium.launch()
         except Exception as e:
             print(repr(e))
             return
+        page = browser.new_page()
+        user_agent = page.evaluate("navigator.userAgent")
+        print(f"User Agent: {user_agent}")
-        page = browser.new_page()
         page.goto(url)
         content = page.content()
         browser.close()

     return content
```
This commit is contained in:
Paul Gauthier 2024-02-08 11:02:54 -08:00
parent 65111591d7
commit f10a9d2e7b

View file

@ -17,6 +17,8 @@ def scrape_with_playwright(url):
# user_agent = ??
page = browser.new_page()
user_agent = page.evaluate("navigator.userAgent")
print(f"User Agent: {user_agent}")
page.goto(url)
content = page.content()
browser.close()