[BUG] Test are breaking for restricted access to the MIND blob

[miguelgfierro]

Description

tmp_path = PosixPath('/tmp/pytest-of-root/pytest-0/test_download_mind_large0')

    def test_download_mind_large(tmp_path):
>       train_path, valid_path = download_mind(size="large", dest_path=tmp_path)

tests/data_validation/recommenders/datasets/test_mind.py:128: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
recommenders/datasets/mind.py:66: in download_mind
    train_path = maybe_download(url=url_train, work_directory=path)
/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/retrying.py:56: in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/retrying.py:266: in call
    raise attempt.get()
/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/retrying.py:301: in get
    six.reraise(self.value[0], self.value[1], self.value[2])
/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/six.py:719: in reraise
    raise value
/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/retrying.py:251: in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
recommenders/datasets/download_utils.py:52: in maybe_download
    r.raise_for_status()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Response [409]>

    def raise_for_status(self):
        """Raises :class:`HTTPError`, if one occurred."""
    
        http_error_msg = ""
        if isinstance(self.reason, bytes):
            # We attempt to decode utf-8 first because some servers
            # choose to localize their reason strings. If the string
            # isn't utf-8, we fall back to iso-8859-1 for all other
            # encodings. (See PR #3538)
            try:
                reason = self.reason.decode("utf-8")
            except UnicodeDecodeError:
                reason = self.reason.decode("iso-8859-1")
        else:
            reason = self.reason
    
        if 400 <= self.status_code < 500:
            http_error_msg = (
                f"{self.status_code} Client Error: {reason} for url: {self.url}"
            )
    
        elif 500 <= self.status_code < 600:
            http_error_msg = (
                f"{self.status_code} Server Error: {reason} for url: {self.url}"
            )
    
        if http_error_msg:
>           raise HTTPError(http_error_msg, response=self)
E           requests.exceptions.HTTPError: 409 Client Error: Public access is not permitted on this storage account. for url: https://mind201910small.blob.core.windows.net/release/MINDlarge_train.zip

/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/requests/models.py:1024: HTTPError
------------------------------ Captured log call -------------------------------
ERROR    recommenders.datasets.download_utils:download_utils.py:51 Problem downloading https://mind201910small.blob.core.windows.net/release/MINDlarge_train.zip
ERROR    recommenders.datasets.download_utils:download_utils.py:51 Problem downloading https://mind201910small.blob.core.windows.net/release/MINDlarge_train.zip
ERROR    recommenders.datasets.download_utils:download_utils.py:51 Problem downloading https://mind201910small.blob.core.windows.net/release/MINDlarge_train.zip
ERROR    recommenders.datasets.download_utils:download_utils.py:51 Problem downloading https://mind201910small.blob.core.windows.net/release/MINDlarge_train.zip
ERROR    recommenders.datasets.download_utils:download_utils.py:51 Problem downloading https://mind201910small.blob.core.windows.net/release/MINDlarge_train.zip
___________________________ test_extract_mind_large ____________________________

tmp = '/tmp/pytest-of-root/pytest-0/tmptwsi8df7'

    def test_extract_mind_large(tmp):
>       train_zip, valid_zip = download_mind(size="large", dest_path=tmp)

tests/data_validation/recommenders/datasets/test_mind.py:136: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
recommenders/datasets/mind.py:66: in download_mind
    train_path = maybe_download(url=url_train, work_directory=path)
/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/retrying.py:56: in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/retrying.py:266: in call
    raise attempt.get()
/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/retrying.py:301: in get
    six.reraise(self.value[0], self.value[1], self.value[2])
/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/six.py:719: in reraise
    raise value
/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/retrying.py:251: in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
recommenders/datasets/download_utils.py:52: in maybe_download
    r.raise_for_status()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Response [409]>

    def raise_for_status(self):
        """Raises :class:`HTTPError`, if one occurred."""
    
        http_error_msg = ""
        if isinstance(self.reason, bytes):
            # We attempt to decode utf-8 first because some servers
            # choose to localize their reason strings. If the string
            # isn't utf-8, we fall back to iso-8859-1 for all other
            # encodings. (See PR #3538)
            try:
                reason = self.reason.decode("utf-8")
            except UnicodeDecodeError:
                reason = self.reason.decode("iso-8859-1")
        else:
            reason = self.reason
    
        if 400 <= self.status_code < 500:
            http_error_msg = (
                f"{self.status_code} Client Error: {reason} for url: {self.url}"
            )
    
        elif 500 <= self.status_code < 600:
            http_error_msg = (
                f"{self.status_code} Server Error: {reason} for url: {self.url}"
            )
    
        if http_error_msg:
>           raise HTTPError(http_error_msg, response=self)
E           requests.exceptions.HTTPError: 409 Client Error: Public access is not permitted on this storage account. for url: https://mind201910small.blob.core.windows.net/release/MINDlarge_train.zip

/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/requests/models.py:1024: HTTPError
------------------------------ Captured log call -------------------------------
ERROR    recommenders.datasets.download_utils:download_utils.py:51 Problem downloading https://mind201910small.blob.core.windows.net/release/MINDlarge_train.zip
ERROR    recommenders.datasets.download_utils:download_utils.py:51 Problem downloading https://mind201910small.blob.core.windows.net/release/MINDlarge_train.zip
ERROR    recommenders.datasets.download_utils:download_utils.py:51 Problem downloading https://mind201910small.blob.core.windows.net/release/MINDlarge_train.zip
ERROR    recommenders.datasets.download_utils:download_utils.py:51 Problem downloading https://mind201910small.blob.core.windows.net/release/MINDlarge_train.zip
ERROR    recommenders.datasets.download_utils:download_utils.py:51 Problem downloading https://mind201910small.blob.core.windows.net/release/MINDlarge_train.zip
_____________________________ test_mind_utils_runs _____________________________

notebooks = {'als_deep_dive': '/mnt/azureml/cr/j/0ff71de6470e4f4eb33b284cd911e6c9/exe/wd/examples/02_model_collaborative_filtering...rk_movielens': '/mnt/azureml/cr/j/0ff71de6470e4f4eb33b284cd911e6c9/exe/wd/examples/06_benchmarks/movielens.ipynb', ...}
output_notebook = 'output.ipynb', kernel_name = 'python3'
tmp = '/tmp/pytest-of-root/pytest-0/tmpnb2e_a7h'

    def test_mind_utils_runs(notebooks, output_notebook, kernel_name, tmp):
        notebook_path = notebooks["mind_utils"]
>       execute_notebook(
            notebook_path,
            output_notebook,
            kernel_name=kernel_name,
            parameters=dict(mind_type="small", word_embedding_dim=300),
        )

tests/data_validation/examples/test_mind.py:9: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
recommenders/utils/notebook_utils.py:102: in execute_notebook
    executed_notebook, _ = execute_preprocessor.preprocess(
/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/nbconvert/preprocessors/execute.py:103: in preprocess
    self.preprocess_cell(cell, resources, index)
/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/nbconvert/preprocessors/execute.py:124: in preprocess_cell
    cell = self.execute_cell(cell, index, store_history=True)
/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/jupyter_core/utils/__init__.py:165: in wrapped
    return loop.run_until_complete(inner)
/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/asyncio/base_events.py:649: in run_until_complete
    return future.result()
/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/nbclient/client.py:1062: in async_execute_cell
    await self._check_raise_for_error(cell, cell_index, exec_reply)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <nbconvert.preprocessors.execute.ExecutePreprocessor object at 0x145c1b978d00>
cell = {'cell_type': 'code', 'execution_count': 3, 'metadata': {'execution': {'iopub.status.busy': '2024-08-01T02:06:38.72166...lid'), clean_zip_file=False)\noutput_path = os.path.join(data_path, 'utils')\nos.makedirs(output_path, exist_ok=True)"}
cell_index = 4
exec_reply = {'buffers': [], 'content': {'ename': 'HTTPError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'ff776c0c-0b2d-4c00-...e, 'engine': 'ff776c0c-0b2d-4c00-8284-f504cc9cd4c9', 'started': '2024-08-01T02:06:38.722124Z', 'status': 'error'}, ...}

    async def _check_raise_for_error(
        self, cell: NotebookNode, cell_index: int, exec_reply: dict[str, t.Any] | None
    ) -> None:
        if exec_reply is None:
            return None
    
        exec_reply_content = exec_reply["content"]
        if exec_reply_content["status"] != "error":
            return None
    
        cell_allows_errors = (not self.force_raise_errors) and (
            self.allow_errors
            or exec_reply_content.get("ename") in self.allow_error_names
            or "raises-exception" in cell.metadata.get("tags", [])
        )
        await run_hook(
            self.on_cell_error, cell=cell, cell_index=cell_index, execute_reply=exec_reply
        )
        if not cell_allows_errors:
>           raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E           nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E           ------------------
E           tmpdir = TemporaryDirectory()
E           data_path = tmpdir.name
E           train_zip, valid_zip = download_mind(size=mind_type, dest_path=data_path)
E           unzip_file(train_zip, os.path.join(data_path, 'train'), clean_zip_file=False)
E           unzip_file(valid_zip, os.path.join(data_path, 'valid'), clean_zip_file=False)
E           output_path = os.path.join(data_path, 'utils')
E           os.makedirs(output_path, exist_ok=True)
E           ------------------
E           
E           ----- stderr -----
E           Problem downloading https://mind201910small.blob.core.windows.net/release/MINDsmall_train.zip
E           ----- stderr -----
E           Problem downloading https://mind201910small.blob.core.windows.net/release/MINDsmall_train.zip
E           ----- stderr -----
E           Problem downloading https://mind201910small.blob.core.windows.net/release/MINDsmall_train.zip
E           ----- stderr -----
E           Problem downloading https://mind201910small.blob.core.windows.net/release/MINDsmall_train.zip
E           ----- stderr -----
E           Problem downloading https://mind201910small.blob.core.windows.net/release/MINDsmall_train.zip
E           ------------------
E           
E           �[0;31m---------------------------------------------------------------------------�[0m
E           �[0;31mHTTPError�[0m                                 Traceback (most recent call last)
E           Cell �[0;32mIn[3], line 3�[0m
E           �[1;32m      1�[0m tmpdir �[38;5;241m=�[39m TemporaryDirectory()
E           �[1;32m      2�[0m data_path �[38;5;241m=�[39m tmpdir�[38;5;241m.�[39mname
E           �[0;32m----> 3�[0m train_zip, valid_zip �[38;5;241m=�[39m �[43mdownload_mind�[49m�[43m(�[49m�[43msize�[49m�[38;5;241;43m=�[39;49m�[43mmind_type�[49m�[43m,�[49m�[43m �[49m�[43mdest_path�[49m�[38;5;241;43m=�[39;49m�[43mdata_path�[49m�[43m)�[49m
E           �[1;32m      4�[0m unzip_file(train_zip, os�[38;5;241m.�[39mpath�[38;5;241m.�[39mjoin(data_path, �[38;5;124m'�[39m�[38;5;124mtrain�[39m�[38;5;124m'�[39m), clean_zip_file�[38;5;241m=�[39m�[38;5;28;01mFalse�[39;00m)
E           �[1;32m      5�[0m unzip_file(valid_zip, os�[38;5;241m.�[39mpath�[38;5;241m.�[39mjoin(data_path, �[38;5;124m'�[39m�[38;5;124mvalid�[39m�[38;5;124m'�[39m), clean_zip_file�[38;5;241m=�[39m�[38;5;28;01mFalse�[39;00m)
E           
E           File �[0;32m/mnt/azureml/cr/j/0ff71de6470e4f4eb33b284cd911e6c9/exe/wd/recommenders/datasets/mind.py:66�[0m, in �[0;36mdownload_mind�[0;34m(size, dest_path)�[0m
E           �[1;32m     64�[0m url_train, url_valid �[38;5;241m=�[39m URL_MIND[size]
E           �[1;32m     65�[0m �[38;5;28;01mwith�[39;00m download_path(dest_path) �[38;5;28;01mas�[39;00m path:
E           �[0;32m---> 66�[0m     train_path �[38;5;241m=�[39m �[43mmaybe_download�[49m�[43m(�[49m�[43murl�[49m�[38;5;241;43m=�[39;49m�[43murl_train�[49m�[43m,�[49m�[43m �[49m�[43mwork_directory�[49m�[38;5;241;43m=�[39;49m�[43mpath�[49m�[43m)�[49m
E           �[1;32m     67�[0m     valid_path �[38;5;241m=�[39m maybe_download(url�[38;5;241m=�[39murl_valid, work_directory�[38;5;241m=�[39mpath)
E           �[1;32m     68�[0m �[38;5;28;01mreturn�[39;00m train_path, valid_path
E           
E           File �[0;32m/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/retrying.py:56�[0m, in �[0;36mretry.<locals>.wrap.<locals>.wrapped_f�[0;34m(*args, **kw)�[0m
E           �[1;32m     54�[0m �[38;5;129m@six�[39m�[38;5;241m.�[39mwraps(f)
E           �[1;32m     55�[0m �[38;5;28;01mdef�[39;00m �[38;5;21mwrapped_f�[39m(�[38;5;241m*�[39margs, �[38;5;241m*�[39m�[38;5;241m*�[39mkw):
E           �[0;32m---> 56�[0m     �[38;5;28;01mreturn�[39;00m �[43mRetrying�[49m�[43m(�[49m�[38;5;241;43m*�[39;49m�[43mdargs�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m*�[39;49m�[38;5;241;43m*�[39;49m�[43mdkw�[49m�[43m)�[49m�[38;5;241;43m.�[39;49m�[43mcall�[49m�[43m(�[49m�[43mf�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m*�[39;49m�[43margs�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m*�[39;49m�[38;5;241;43m*�[39;49m�[43mkw�[49m�[43m)�[49m
E           
E           File �[0;32m/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/retrying.py:266�[0m, in �[0;36mRetrying.call�[0;34m(self, fn, *args, **kwargs)�[0m
E           �[1;32m    263�[0m �[38;5;28;01mif�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39mstop(attempt_number, delay_since_first_attempt_ms):
E           �[1;32m    264�[0m     �[38;5;28;01mif�[39;00m �[38;5;129;01mnot�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39m_wrap_exception �[38;5;129;01mand�[39;00m attempt�[38;5;241m.�[39mhas_exception:
E           �[1;32m    265�[0m         �[38;5;66;03m# get() on an attempt with an exception should cause it to be raised, but raise just in case�[39;00m
E           �[0;32m--> 266�[0m         �[38;5;28;01mraise�[39;00m �[43mattempt�[49m�[38;5;241;43m.�[39;49m�[43mget�[49m�[43m(�[49m�[43m)�[49m
E           �[1;32m    267�[0m     �[38;5;28;01melse�[39;00m:
E           �[1;32m    268�[0m         �[38;5;28;01mraise�[39;00m RetryError(attempt)
E           
E           File �[0;32m/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/retrying.py:301�[0m, in �[0;36mAttempt.get�[0;34m(self, wrap_exception)�[0m
E           �[1;32m    299�[0m         �[38;5;28;01mraise�[39;00m RetryError(�[38;5;28mself�[39m)
E           �[1;32m    300�[0m     �[38;5;28;01melse�[39;00m:
E           �[0;32m--> 301�[0m         �[43msix�[49m�[38;5;241;43m.�[39;49m�[43mreraise�[49m�[43m(�[49m�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43mvalue�[49m�[43m[�[49m�[38;5;241;43m0�[39;49m�[43m]�[49m�[43m,�[49m�[43m �[49m�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43mvalue�[49m�[43m[�[49m�[38;5;241;43m1�[39;49m�[43m]�[49m�[43m,�[49m�[43m �[49m�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43mvalue�[49m�[43m[�[49m�[38;5;241;43m2�[39;49m�[43m]�[49m�[43m)�[49m
E           �[1;32m    302�[0m �[38;5;28;01melse�[39;00m:
E           �[1;32m    303�[0m     �[38;5;28;01mreturn�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39mvalue
E           
E           File �[0;32m/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/six.py:719�[0m, in �[0;36mreraise�[0;34m(tp, value, tb)�[0m
E           �[1;32m    717�[0m     �[38;5;28;01mif�[39;00m value�[38;5;241m.�[39m__traceback__ �[38;5;129;01mis�[39;00m �[38;5;129;01mnot�[39;00m tb:
E           �[1;32m    718�[0m         �[38;5;28;01mraise�[39;00m value�[38;5;241m.�[39mwith_traceback(tb)
E           �[0;32m--> 719�[0m     �[38;5;28;01mraise�[39;00m value
E           �[1;32m    720�[0m �[38;5;28;01mfinally�[39;00m:
E           �[1;32m    721�[0m     value �[38;5;241m=�[39m �[38;5;28;01mNone�[39;00m
E           
E           File �[0;32m/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/retrying.py:251�[0m, in �[0;36mRetrying.call�[0;34m(self, fn, *args, **kwargs)�[0m
E           �[1;32m    248�[0m     �[38;5;28mself�[39m�[38;5;241m.�[39m_before_attempts(attempt_number)
E           �[1;32m    250�[0m �[38;5;28;01mtry�[39;00m:
E           �[0;32m--> 251�[0m     attempt �[38;5;241m=�[39m Attempt(�[43mfn�[49m�[43m(�[49m�[38;5;241;43m*�[39;49m�[43margs�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m*�[39;49m�[38;5;241;43m*�[39;49m�[43mkwargs�[49m�[43m)�[49m, attempt_number, �[38;5;28;01mFalse�[39;00m)
E           �[1;32m    252�[0m �[38;5;28;01mexcept�[39;00m:
E           �[1;32m    253�[0m     tb �[38;5;241m=�[39m sys�[38;5;241m.�[39mexc_info()
E           
E           File �[0;32m/mnt/azureml/cr/j/0ff71de6470e4f4eb33b284cd911e6c9/exe/wd/recommenders/datasets/download_utils.py:52�[0m, in �[0;36mmaybe_download�[0;34m(url, filename, work_directory, expected_bytes)�[0m
E           �[1;32m     50�[0m     �[38;5;28;01melse�[39;00m:
E           �[1;32m     51�[0m         log�[38;5;241m.�[39merror(�[38;5;124mf�[39m�[38;5;124m"�[39m�[38;5;124mProblem downloading �[39m�[38;5;132;01m{�[39;00murl�[38;5;132;01m}�[39;00m�[38;5;124m"�[39m)
E           �[0;32m---> 52�[0m         �[43mr�[49m�[38;5;241;43m.�[39;49m�[43mraise_for_status�[49m�[43m(�[49m�[43m)�[49m
E           �[1;32m     53�[0m �[38;5;28;01melse�[39;00m:
E           �[1;32m     54�[0m     log�[38;5;241m.�[39minfo(�[38;5;124mf�[39m�[38;5;124m"�[39m�[38;5;124mFile �[39m�[38;5;132;01m{�[39;00mfilepath�[38;5;132;01m}�[39;00m�[38;5;124m already downloaded�[39m�[38;5;124m"�[39m)
E           
E           File �[0;32m/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/requests/models.py:1024�[0m, in �[0;36mResponse.raise_for_status�[0;34m(self)�[0m
E           �[1;32m   1019�[0m     http_error_msg �[38;5;241m=�[39m (
E           �[1;32m   1020�[0m         �[38;5;124mf�[39m�[38;5;124m"�[39m�[38;5;132;01m{�[39;00m�[38;5;28mself�[39m�[38;5;241m.�[39mstatus_code�[38;5;132;01m}�[39;00m�[38;5;124m Server Error: �[39m�[38;5;132;01m{�[39;00mreason�[38;5;132;01m}�[39;00m�[38;5;124m for url: �[39m�[38;5;132;01m{�[39;00m�[38;5;28mself�[39m�[38;5;241m.�[39murl�[38;5;132;01m}�[39;00m�[38;5;124m"�[39m
E           �[1;32m   1021�[0m     )
E           �[1;32m   1023�[0m �[38;5;28;01mif�[39;00m http_error_msg:
E           �[0;32m-> 1024�[0m     �[38;5;28;01mraise�[39;00m HTTPError(http_error_msg, response�[38;5;241m=�[39m�[38;5;28mself�[39m)
E           
E           �[0;31mHTTPError�[0m: 409 Client Error: Public access is not permitted on this storage account. for url: https://mind201910small.blob.core.windows.net/release/MINDsmall_train.zip

/azureml-envs/azureml_ee05c39251ad373fce7f4861d6ed56fe/lib/python3.10/site-packages/nbclient/client.py:918: CellExecutionError

In which platform does it happen?

How do we replicate the issue?

https://github.com/recommenders-team/recommenders/actions/runs/13447074780/job/37577703132

Expected behavior (i.e. solution)

Willingness to contribute

• Yes, I can contribute for this issue independently.
• Yes, I can contribute for this issue with guidance from Recommenders community.
• No, I cannot contribute at this time.

Other Comments

[miguelgfierro]

In local it works:

$ pytest tests/data_validation/recommenders/datasets/test_mind.py 
======================================================================== test session starts ========================================================================
platform linux -- Python 3.11.9, pytest-8.2.2, pluggy-1.5.0
rootdir: /home/miguel/MS/recommenders
configfile: pyproject.toml
plugins: anyio-4.4.0, cov-5.0.0, typeguard-4.3.0, hypothesis-6.104.2, mock-3.14.0
collected 15 items                                                                                                                                                  

tests/data_validation/recommenders/datasets/test_mind.py ...............                                                                                      [100%]

================================================================== 15 passed in 204.00s (0:03:23) ===================================================================

This is weird @SimonYansenZhao @anargyri do you think it could be because there is some kind of restrictions on Azure to download that dataset?

Can you check if it works for you: pytest tests/data_validation/recommenders/datasets/test_mind.py