LangChain DocumentLoaders – Hamel’s Blog - Hamel Husain

LangChain — это библиотека, предоставляющая целый набор инструментов для LLM, в частности для интеграции LLM с другими инструментами.

Одна из недооценённых возможностей Langchain — это DocumentLoaders, которые позволяют получать текстовые данные из любого источника, что чрезвычайно полезно, даже если вы вообще не используете LLM! (Эти загрузчики также можно «приспособить» для сбора данных для дообучения!)

Например, если вы пытаетесь получить данные с сайта в виде текста, вот несколько полезных DocumentLoaders:

RecursiveURLLoader SeleniumLoader SitemapLoader: он рассматривается ниже.

Я считаю полезным сочетать DocumentLoaders из LangChain с датасетами HuggingFace, потому что это позволяет сохранять, версионировать и делать другие интересные вещи, например выполнять семантический поиск по вашим данным с помощью FAISS.

На момент написания существует более 125 различных видов DocumentLoaders. Мне ещё не доводилось не найти подходящий загрузчик для быстрого получения нужных данных.

Sitemap Loader

Карты сайта (Sitemaps) — удобный способ увидеть перечень всех страниц сайта. Это полезно для сбора всего текста с большого сайта, который может содержать множество страниц. Ниже я использую SitemapLoader, чтобы получить весь текст с https://quarto.org.

В настоящее время в langchain есть баг, поэтому мне пришлось установить старую версию, предшествующую этому коммиту, который сломал SitemapLoader. Мне пришлось откатиться до v0.0.202 через pip install langchain==0.0.202

import nest_asyncio nest_asyncio.apply() # you don't need this line outside notebooks from langchain.document_loaders.sitemap import SitemapLoader sitemap_loader = SitemapLoader(web_path="https://quarto.org/sitemap.xml") sitemap_loader.requests_per_second = 4 docs = sitemap_loader.load()

Fetching pages: 100%|####################################| 269/269 [00:16<00:00, 16.21it/s]

print(f'There are {len(docs)} pages')

There are 269 pages

Давайте посмотрим на содержимое одной страницы:

example = docs[0] example.dict()

{'page_content': '\n\n\n\n\nQuarto - About Quarto\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nOverview\n\n\n\nGet Started\n\n\n\nGuide\n\n\n\nExtensions\n\n\n\nReference\n\n\n\nGallery\n\n\n\nBlog\n\n\n\nHelp\n\n\n\n\n\nReport a Bug\n\n\n\n\nAsk a Question\n\n\n\n\nFAQ\n\n\n\n\n \n\n\n\n\n\n \n\n\n\n\n\n\n\n\nOn this page\n\nGoals\nProject\nContribute\n\nEdit this pageReport an issue\n\n\n\n\n\nAbout Quarto\nOpen source tools for scientific and technical publishing\n\n\n\n\n\nGoals\nThe overarching goal of Quarto is to make the process of creating and collaborating on scientific and technical documents dramatically better. We hope to do this in several dimensions:\n\nCreate a writing and publishing environment with great integrated tools for technical content. We want to make authoring with embedded code, equations, figures, complex diagrams, interactive widgets, citations, cross references, and the myriad other special requirements of scientific discourse straightforward and productive for everyone.\nHelp authors take full advantage of the web as a connected, interactive platform for communications, while still providing the ability to create excellent printed output from the same document source. Researchers shouldn’t need to choose between LaTeX, MS Word, and HTML but rather be able to author documents that target all of them at the same time.\nMake reproducible research and publications the norm rather than the exception. Reproducibility requires that the code and data required to create a manuscript are an integrated part of it. However, this isn’t often straightforward in practice—Quarto aims to make it easier to adopt a reproducible workflow than not.\n\nQuarto is open source software licensed under the GNU GPL v2. We believe that it’s better for everyone if the tools used for research and science are free and open. Reproducibility, widespread sharing of knowledge and techniques, and the leveling of the playing field by eliminating cost barriers are but a few of the shared benefits of free software in science.\n\n\nProject\nAt the core of Quarto is Pandoc, a powerful and flexible document processing tool. Quarto adds a number of facilities to Pandoc aimed at scientific and technical publishing, including:\n\nEmbedding code and output from Python, R, and JavaScript via integration with Jupyter, Knitr, and Observable.\nA variety of extensions to Pandoc markdown useful for technical writing including cross-references, sub-figures, layout panels, hoverable citations and footnotes, callouts, and more.\nA project system for rendering groups of documents at once, sharing options across documents, and producing aggregate output like websites and books.\n\nDevelopment of Quarto is sponsored by Posit, PBC, where we previously created a similar system (R Markdown) that shared the same goals, but was targeted principally at users of the R language. The same core team works on both Quarto and R Markdown:\n\nJ.J. Allaire (@jjallaire)\nChristophe Dervieux (@cderv)\nCarlos Scheidegger (@cscheid)\nCharles Teague (@dragonstyle)\nYihui Xie (@yihui)\n\nWith Quarto, we are hoping to bring these tools to a much wider audience.\nQuarto is a registered trademark of Posit. Please see our trademark policy for guidelines on usage of the Quarto trademark.\n\n\nContribute\nYou can contribute to Quarto in many ways:\n\nBy opening issues to provide feedback and share ideas.\nBy submitting Pull Request (PR) to fix opened issues\nBy submitting Pull Request (PR) to suggest new features (it is considered good practice to open an issue for discussion before working on a pull request for a new feature).\n\nPlease be mindful of our code of conduct as you interact with other community members.\n\nPull Requests\nPull requests are very welcome! Here’s how to contribute via PR:\n\nFork the repository, clone it locally, and make your changes in a new branch specific to the PR. For example:\n\n\nTerminal\n\n# clone your fork\n$ git clone https://github.com//quarto-cli\n\n# configure for your platform (./configure.sh or ./configure.cmd for windows)\n$ cd quarto-cli\n$ ./configure.sh\n\n# checkout a new branch\n$ git checkout -b feature/newthing\n\nFor significant changes (e.g more than small bug fixes), ensure that you have signed the individual or corporate contributor agreement as appropriate. You can send the signed copy to jj@rstudio.com.\nSubmit the pull request. It is ok to submit as draft in your are still working on it but would like some feedback from us. It always good to share in the open that you are working on it.\n\nWe’ll try to be as responsive as possible in reviewing and accepting pull requests.\n\n\n \n\n\n \n\n\n\nProudly supported by \n\n\n\n\n\nAbout\n\n\n\n\nFAQ\n\n\n\n\nLicense\n\n\n\n\nTrademark\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n', 'metadata': {'source': 'https://quarto.org/about.html', 'loc': 'https://quarto.org/about.html', 'lastmod': '2023-07-05T19:35:15.135Z'}}

Очистка данных

Когда мы смотрим на эту страницу, видно много лишнего текста. Появляются навбар и боковая навигация, а они нам не нужны. Мы можем обновить функцию парсинга, чтобы это исправить:

from bs4 import BeautifulSoup def remove_nav_and_header_elements(content: BeautifulSoup) -> str: exclude = content.find_all(["nav", "footer", "header", "head"]) for element in exclude: element.decompose() return str(content.get_text()).strip()

sitemap_loader = SitemapLoader(web_path="https://quarto.org/sitemap.xml", parsing_function=remove_nav_and_header_elements) sitemap_loader.requests_per_second = 4 docs = sitemap_loader.load()

Fetching pages: 100%|####################################| 269/269 [00:05<00:00, 52.00it/s]

example = docs[0] example.dict()

{'page_content': 'Goals\nThe overarching goal of Quarto is to make the process of creating and collaborating on scientific and technical documents dramatically better. We hope to do this in several dimensions:\n\nCreate a writing and publishing environment with great integrated tools for technical content. We want to make authoring with embedded code, equations, figures, complex diagrams, interactive widgets, citations, cross references, and the myriad other special requirements of scientific discourse straightforward and productive for everyone.\nHelp authors take full advantage of the web as a connected, interactive platform for communications, while still providing the ability to create excellent printed output from the same document source. Researchers shouldn’t need to choose between LaTeX, MS Word, and HTML but rather be able to author documents that target all of them at the same time.\nMake reproducible research and publications the norm rather than the exception. Reproducibility requires that the code and data required to create a manuscript are an integrated part of it. However, this isn’t often straightforward in practice—Quarto aims to make it easier to adopt a reproducible workflow than not.\n\nQuarto is open source software licensed under the GNU GPL v2. We believe that it’s better for everyone if the tools used for research and science are free and open. Reproducibility, widespread sharing of knowledge and techniques, and the leveling of the playing field by eliminating cost barriers are but a few of the shared benefits of free software in science.\n\n\nProject\nAt the core of Quarto is Pandoc, a powerful and flexible document processing tool. Quarto adds a number of facilities to Pandoc aimed at scientific and technical publishing, including:\n\nEmbedding code and output from Python, R, and JavaScript via integration with Jupyter, Knitr, and Observable.\nA variety of extensions to Pandoc markdown useful for technical writing including cross-references, sub-figures, layout panels, hoverable citations and footnotes, callouts, and more.\nA project system for rendering groups of documents at once, sharing options across documents, and producing aggregate output like websites and books.\n\nDevelopment of Quarto is sponsored by Posit, PBC, where we previously created a similar system (R Markdown) that shared the same goals, but was targeted principally at users of the R language. The same core team works on both Quarto and R Markdown:\n\nJ.J. Allaire (@jjallaire)\nChristophe Dervieux (@cderv)\nCarlos Scheidegger (@cscheid)\nCharles Teague (@dragonstyle)\nYihui Xie (@yihui)\n\nWith Quarto, we are hoping to bring these tools to a much wider audience.\nQuarto is a registered trademark of Posit. Please see our trademark policy for guidelines on usage of the Quarto trademark.\n\n\nContribute\nYou can contribute to Quarto in many ways:\n\nBy opening issues to provide feedback and share ideas.\nBy submitting Pull Request (PR) to fix opened issues\nBy submitting Pull Request (PR) to suggest new features (it is considered good practice to open an issue for discussion before working on a pull request for a new feature).\n\nPlease be mindful of our code of conduct as you interact with other community members.\n\nPull Requests\nPull requests are very welcome! Here’s how to contribute via PR:\n\nFork the repository, clone it locally, and make your changes in a new branch specific to the PR. For example:\n\n\nTerminal\n\n# clone your fork\n$ git clone https://github.com//quarto-cli\n\n# configure for your platform (./configure.sh or ./configure.cmd for windows)\n$ cd quarto-cli\n$ ./configure.sh\n\n# checkout a new branch\n$ git checkout -b feature/newthing\n\nFor significant changes (e.g more than small bug fixes), ensure that you have signed the individual or corporate contributor agreement as appropriate. You can send the signed copy to jj@rstudio.com.\nSubmit the pull request. It is ok to submit as draft in your are still working on it but would like some feedback from us. It always good to share in the open that you are working on it.\n\nWe’ll try to be as responsive as possible in reviewing and accepting pull requests.', 'metadata': {'source': 'https://quarto.org/about.html', 'loc': 'https://quarto.org/about.html', 'lastmod': '2023-07-05T19:35:15.135Z'}}

Создание датасета HF

Мы можем использовать метод from_list, чтобы загрузить эти данные карты сайта в датасет HF.

from datasets import Dataset repo_name = 'hamel/quarto'

quarto_data = Dataset.from_list([d.dict() for d in docs])

quarto_data

Dataset({ features: ['page_content', 'metadata'], num_rows: 269 })

quarto_data.push_to_hub(repo_name)

Updating downloaded metadata with the new split.

Эти данные доступны по адресу https://huggingface.co/datasets/hamel/quarto

Загрузка данных

Вы можете скачать данные с HuggingFace Hub следующим образом:

from datasets import load_dataset remote_data = load_dataset(repo_name)

Using custom data configuration hamel--quarto-b88699e31e28f953

Downloading and preparing dataset None/None (download: Unknown size, generated: 1.81 MiB, post-processed: Unknown size, total: 1.81 MiB) to /Users/hamel/.cache/huggingface/datasets/hamel___parquet/hamel--quarto-b88699e31e28f953/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec...

Dataset parquet downloaded and prepared to /Users/hamel/.cache/huggingface/datasets/hamel___parquet/hamel--quarto-b88699e31e28f953/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec. Subsequent calls will reuse this data.

remote_data['train'][0]

{'page_content': 'Goals\nThe overarching goal of Quarto is to make the process of creating and collaborating on scientific and technical documents dramatically better. We hope to do this in several dimensions:\n\nCreate a writing and publishing environment with great integrated tools for technical content. We want to make authoring with embedded code, equations, figures, complex diagrams, interactive widgets, citations, cross references, and the myriad other special requirements of scientific discourse straightforward and productive for everyone.\nHelp authors take full advantage of the web as a connected, interactive platform for communications, while still providing the ability to create excellent printed output from the same document source. Researchers shouldn’t need to choose between LaTeX, MS Word, and HTML but rather be able to author documents that target all of them at the same time.\nMake reproducible research and publications the norm rather than the exception. Reproducibility requires that the code and data required to create a manuscript are an integrated part of it. However, this isn’t often straightforward in practice—Quarto aims to make it easier to adopt a reproducible workflow than not.\n\nQuarto is open source software licensed under the GNU GPL v2. We believe that it’s better for everyone if the tools used for research and science are free and open. Reproducibility, widespread sharing of knowledge and techniques, and the leveling of the playing field by eliminating cost barriers are but a few of the shared benefits of free software in science.\n\n\nProject\nAt the core of Quarto is Pandoc, a powerful and flexible document processing tool. Quarto adds a number of facilities to Pandoc aimed at scientific and technical publishing, including:\n\nEmbedding code and output from Python, R, and JavaScript via integration with Jupyter, Knitr, and Observable.\nA variety of extensions to Pandoc markdown useful for technical writing including cross-references, sub-figures, layout panels, hoverable citations and footnotes, callouts, and more.\nA project system for rendering groups of documents at once, sharing options across documents, and producing aggregate output like websites and books.\n\nDevelopment of Quarto is sponsored by Posit, PBC, where we previously created a similar system (R Markdown) that shared the same goals, but was targeted principally at users of the R language. The same core team works on both Quarto and R Markdown:\n\nJ.J. Allaire (@jjallaire)\nChristophe Dervieux (@cderv)\nCarlos Scheidegger (@cscheid)\nCharles Teague (@dragonstyle)\nYihui Xie (@yihui)\n\nWith Quarto, we are hoping to bring these tools to a much wider audience.\nQuarto is a registered trademark of Posit. Please see our trademark policy for guidelines on usage of the Quarto trademark.\n\n\nContribute\nYou can contribute to Quarto in many ways:\n\nBy opening issues to provide feedback and share ideas.\nBy submitting Pull Request (PR) to fix opened issues\nBy submitting Pull Request (PR) to suggest new features (it is considered good practice to open an issue for discussion before working on a pull request for a new feature).\n\nPlease be mindful of our code of conduct as you interact with other community members.\n\nPull Requests\nPull requests are very welcome! Here’s how to contribute via PR:\n\nFork the repository, clone it locally, and make your changes in a new branch specific to the PR. For example:\n\n\nTerminal\n\n# clone your fork\n$ git clone https://github.com//quarto-cli\n\n# configure for your platform (./configure.sh or ./configure.cmd for windows)\n$ cd quarto-cli\n$ ./configure.sh\n\n# checkout a new branch\n$ git checkout -b feature/newthing\n\nFor significant changes (e.g more than small bug fixes), ensure that you have signed the individual or corporate contributor agreement as appropriate. You can send the signed copy to jj@rstudio.com.\nSubmit the pull request. It is ok to submit as draft in your are still working on it but would like some feedback from us. It always good to share in the open that you are working on it.\n\nWe’ll try to be as responsive as possible in reviewing and accepting pull requests.', 'metadata': {'lastmod': '2023-07-05T19:35:15.135Z', 'loc': 'https://quarto.org/about.html', 'source': 'https://quarto.org/about.html'}}

GitHub Issues

Мы можем использовать GitHubIssuesLoader, чтобы получить все issues из репозитория GitHub.

from langchain.document_loaders import GitHubIssuesLoader

Предполагается, что вы задали GITHUB_PERSONAL_ACCESS_TOKEN как переменную окружения

loader = GitHubIssuesLoader( repo="quarto-dev/quarto-cli", state='all', #get both open and closed issues include_prs=False, ) quarto_issues = loader.load()

len(quarto_issues)

2841

Ого, это очень много Issues! Давайте посмотрим на одну из них:

В issue ниже видно, что она не включает комментарии. Их пришлось бы получать отдельно дополнительными вызовами API, но это уже хорошее начало!

quarto_issues[0]

Document(page_content="### Bug description\n\nI am running Quarto to produce a report in which I mix python with R (using 95% R). When I started trying to use python, I believe that RStudio is not recognizing my python installation and packages:\r\n\r\n```\r\nError in py_call_impl(callable, call_args$unnamed, call_args$named) : \r\n ModuleNotFoundError: No module named 'pandas'\r\n```\r\n\r\nAny ideas?\n\n### Steps to reproduce\n\n---\r\ntitle: 'Nota Técnica - Índice Socioeconômico dos Estudantes'\r\nauthor: ''\r\nformat: \r\n html:\r\n encoding: 'UTF-8'\r\n theme: style.css\r\neditor: visual\r\nlang: pt\r\nexecute:\r\n echo: false\r\n warning: false\r\n---\r\n\r\n```{python}\r\n#| label: load-py-pckgs\r\nimport pandas as pd\r\nimport seaborn as sns\r\n```\n\n### Expected behavior\n\nI expected it to load python packages and modules with no problems.\n\n### Actual behavior\n\nError when loading any python package.\n\n### Your environment\n\nIDE: RStudio 2022.07.2 Build 576\r\nR version 4.1.0 (2021-05-18) (can't update because of internal packages built with this version)\r\nOS: Windows 11\n\n### Quarto check output\n\n`quarto check` gives me:\r\nC:\\Users\\joao.freire\\Documents>quarto check\r\n\r\n[>] Checking versions of quarto binary dependencies...\r\n Pandoc version 3.1.1: OK\r\n Dart Sass version 1.55.0: OK\r\n[>] Checking versions of quarto dependencies......OK\r\n[>] Checking Quarto installation......OK\r\n Version: 1.3.433\r\n Path: C:\\Users\\joao.freire\\AppData\\Local\\Programs\\Quarto\\bin\r\n CodePage: 1252\r\n\r\n[>] Checking basic markdown render....OK\r\n\r\n[>] Checking Python 3 installation....OK\r\n Version: 3.11.4\r\n Path: C:/Program Files/Python311/python.exe\r\n Jupyter: 5.3.1\r\n Kernels: python3\r\n\r\n(/) Checking Jupyter engine render....0.00s - Debugger warning: It seems that frozen modules are being used, which may\r\n0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off\r\n0.00s - to python to disable frozen modules.\r\n0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this\r\nvalidation.\r\n0.00s - Debugger warning: It seems that frozen modules are being used, which may\r\n0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off\r\n0.00s - to python to disable frozen modules.\r\n0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this\r\nvalidation.\r\n[>] Checking Jupyter engine render....OK\r\n\r\n[>] Checking R installation...........OK\r\n Version: 4.1.0\r\n Path: C:/Users/joao.freire/Documents/R/R-4.1.0\r\n LibPaths:\r\n - C:/Users/joao.freire/Documents/R/R-4.1.0/library\r\n knitr: 1.42\r\n rmarkdown: 2.23\r\n\r\n[>] Checking Knitr engine render......OK\r\n\r\n\r\nC:\\Users\\joao.freire\\Documents>quarto check\r\n\r\n[>] Checking versions of quarto binary dependencies...\r\n Pandoc version 3.1.1: OK\r\n Dart Sass version 1.55.0: OK\r\n[>] Checking versions of quarto dependencies......OK\r\n[>] Checking Quarto installation......OK\r\n Version: 1.3.433\r\n Path: C:\\Users\\joao.freire\\AppData\\Local\\Programs\\Quarto\\bin\r\n CodePage: 1252\r\n\r\n[>] Checking basic markdown render....OK\r\n\r\n[>] Checking Python 3 installation....OK\r\n Version: 3.11.4\r\n Path: C:/Program Files/Python311/python.exe\r\n Jupyter: 5.3.1\r\n Kernels: python3\r\n\r\n```\r\n(\\) Checking Jupyter engine render....0.00s - Debugger warning: It seems that frozen modules are being used, which may\r\n0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off\r\n0.00s - to python to disable frozen modules.\r\n0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this\r\nvalidation.\r\n0.00s - Debugger warning: It seems that frozen modules are being used, which may\r\n0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off\r\n0.00s - to python to disable frozen modules.\r\n0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this\r\nvalidation.\r\n[>] Checking Jupyter engine render....OK\r\n\r\n[>] Checking R installation...........OK\r\n Version: 4.1.0\r\n Path: C:/Users/joao.freire/Documents/R/R-4.1.0\r\n LibPaths:\r\n - C:/Users/joao.freire/Documents/R/R-4.1.0/library\r\n knitr: 1.42\r\n rmarkdown: 2.23\r\n\r\n[>] Checking Knitr engine render......OK\r\n```", metadata={'url': 'https://github.com/quarto-dev/quarto-cli/issues/6113', 'title': 'Quarto not recognizing python packages', 'creator': 'joaoaugustofrei', 'created_at': '2023-07-05T21:11:31Z', 'comments': 1, 'state': 'open', 'labels': ['bug'], 'assignee': None, 'milestone': None, 'locked': False, 'number': 6113, 'is_pull_request': False})

Загрузка в Hub

Мы можем загрузить эти issues в hub вот так — они будут доступны по адресу https://huggingface.co/datasets/hamel/quarto-issues

ds = Dataset.from_list([x.dict() for x in quarto_issues]) ds.push_to_hub('hamel/quarto-issues')

from datasets import load_dataset remote_data = load_dataset('hamel/quarto-issues')

Using custom data configuration hamel--quarto-issues-52921768ee5c97fb

Downloading and preparing dataset None/None (download: 1.99 MiB, generated: 4.78 MiB, post-processed: Unknown size, total: 6.77 MiB) to /Users/hamel/.cache/huggingface/datasets/hamel___parquet/hamel--quarto-issues-52921768ee5c97fb/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec...

Dataset parquet downloaded and prepared to /Users/hamel/.cache/huggingface/datasets/hamel___parquet/hamel--quarto-issues-52921768ee5c97fb/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec. Subsequent calls will reuse this data.

remote_data['train'][0]

{'page_content': "### Bug description\n\nI am running Quarto to produce a report in which I mix python with R (using 95% R). When I started trying to use python, I believe that RStudio is not recognizing my python installation and packages:\r\n\r\n```\r\nError in py_call_impl(callable, call_args$unnamed, call_args$named) : \r\n ModuleNotFoundError: No module named 'pandas'\r\n```\r\n\r\nAny ideas?\n\n### Steps to reproduce\n\n---\r\ntitle: 'Nota Técnica - Índice Socioeconômico dos Estudantes'\r\nauthor: ''\r\nformat: \r\n html:\r\n encoding: 'UTF-8'\r\n theme: style.css\r\neditor: visual\r\nlang: pt\r\nexecute:\r\n echo: false\r\n warning: false\r\n---\r\n\r\n```{python}\r\n#| label: load-py-pckgs\r\nimport pandas as pd\r\nimport seaborn as sns\r\n```\n\n### Expected behavior\n\nI expected it to load python packages and modules with no problems.\n\n### Actual behavior\n\nError when loading any python package.\n\n### Your environment\n\nIDE: RStudio 2022.07.2 Build 576\r\nR version 4.1.0 (2021-05-18) (can't update because of internal packages built with this version)\r\nOS: Windows 11\n\n### Quarto check output\n\n`quarto check` gives me:\r\nC:\\Users\\joao.freire\\Documents>quarto check\r\n\r\n[>] Checking versions of quarto binary dependencies...\r\n Pandoc version 3.1.1: OK\r\n Dart Sass version 1.55.0: OK\r\n[>] Checking versions of quarto dependencies......OK\r\n[>] Checking Quarto installation......OK\r\n Version: 1.3.433\r\n Path: C:\\Users\\joao.freire\\AppData\\Local\\Programs\\Quarto\\bin\r\n CodePage: 1252\r\n\r\n[>] Checking basic markdown render....OK\r\n\r\n[>] Checking Python 3 installation....OK\r\n Version: 3.11.4\r\n Path: C:/Program Files/Python311/python.exe\r\n Jupyter: 5.3.1\r\n Kernels: python3\r\n\r\n(/) Checking Jupyter engine render....0.00s - Debugger warning: It seems that frozen modules are being used, which may\r\n0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off\r\n0.00s - to python to disable frozen modules.\r\n0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this\r\nvalidation.\r\n0.00s - Debugger warning: It seems that frozen modules are being used, which may\r\n0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off\r\n0.00s - to python to disable frozen modules.\r\n0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this\r\nvalidation.\r\n[>] Checking Jupyter engine render....OK\r\n\r\n[>] Checking R installation...........OK\r\n Version: 4.1.0\r\n Path: C:/Users/joao.freire/Documents/R/R-4.1.0\r\n LibPaths:\r\n - C:/Users/joao.freire/Documents/R/R-4.1.0/library\r\n knitr: 1.42\r\n rmarkdown: 2.23\r\n\r\n[>] Checking Knitr engine render......OK\r\n\r\n\r\nC:\\Users\\joao.freire\\Documents>quarto check\r\n\r\n[>] Checking versions of quarto binary dependencies...\r\n Pandoc version 3.1.1: OK\r\n Dart Sass version 1.55.0: OK\r\n[>] Checking versions of quarto dependencies......OK\r\n[>] Checking Quarto installation......OK\r\n Version: 1.3.433\r\n Path: C:\\Users\\joao.freire\\AppData\\Local\\Programs\\Quarto\\bin\r\n CodePage: 1252\r\n\r\n[>] Checking basic markdown render....OK\r\n\r\n[>] Checking Python 3 installation....OK\r\n Version: 3.11.4\r\n Path: C:/Program Files/Python311/python.exe\r\n Jupyter: 5.3.1\r\n Kernels: python3\r\n\r\n```\r\n(\\) Checking Jupyter engine render....0.00s - Debugger warning: It seems that frozen modules are being used, which may\r\n0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off\r\n0.00s - to python to disable frozen modules.\r\n0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this\r\nvalidation.\r\n0.00s - Debugger warning: It seems that frozen modules are being used, which may\r\n0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off\r\n0.00s - to python to disable frozen modules.\r\n0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this\r\nvalidation.\r\n[>] Checking Jupyter engine render....OK\r\n\r\n[>] Checking R installation...........OK\r\n Version: 4.1.0\r\n Path: C:/Users/joao.freire/Documents/R/R-4.1.0\r\n LibPaths:\r\n - C:/Users/joao.freire/Documents/R/R-4.1.0/library\r\n knitr: 1.42\r\n rmarkdown: 2.23\r\n\r\n[>] Checking Knitr engine render......OK\r\n```", 'metadata': {'assignee': None, 'comments': 1, 'created_at': '2023-07-05T21:11:31Z', 'creator': 'joaoaugustofrei', 'is_pull_request': False, 'labels': ['bug'], 'locked': False, 'milestone': None, 'number': 6113, 'state': 'open', 'title': 'Quarto not recognizing python packages', 'url': 'https://github.com/quarto-dev/quarto-cli/issues/6113'}}