(llava) zl_swuer@Zhengli:~/SQ-LLaVA-main$ pip install flash-attn --no-build-isolation -i https://pypi.tuna.tsinghua.edu.cn/simple --trusted-host pypi.tuna.tsinghua.edu.cn
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting flash-attn
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/11/34/9bf60e736ed7bbe15055ac2dab48ec67d9dbd088d2b4ae318fd77190ab4e/flash_attn-2.7.4.post1.tar.gz (6.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.0/6.0 MB 18.4 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Requirement already satisfied: torch in /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages (from flash-attn) (2.0.1+cu117)
Requirement already satisfied: einops in /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages (from flash-attn) (0.6.1)
Requirement already satisfied: filelock in /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages (from torch->flash-attn) (3.18.0)
Requirement already satisfied: typing-extensions in /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages (from torch->flash-attn) (4.13.2)
Requirement already satisfied: sympy in /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages (from torch->flash-attn) (1.14.0)
Requirement already satisfied: networkx in /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages (from torch->flash-attn) (3.4.2)
Requirement already satisfied: jinja2 in /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages (from torch->flash-attn) (3.1.6)
Requirement already satisfied: triton==2.0.0 in /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages (from torch->flash-attn) (2.0.0)
Requirement already satisfied: cmake in /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages (from triton==2.0.0->torch->flash-attn) (4.0.2)
Requirement already satisfied: lit in /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages (from triton==2.0.0->torch->flash-attn) (18.1.8)
Requirement already satisfied: MarkupSafe>=2.0 in /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages (from jinja2->torch->flash-attn) (2.1.5)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages (from sympy->torch->flash-attn) (1.3.0)
Building wheels for collected packages: flash-attn
DEPRECATION: Building 'flash-attn' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build-isolation`), or adding a `pyproject.toml` file to the source tree of 'flash-attn'. Discussion can be found at https://github.com/pypa/pip/issues/6334
Building wheel for flash-attn (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [267 lines of output]
No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda-11.7'
torch.__version__ = 2.0.1+cu117
/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/__init__.py:94: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!
********************************************************************************
Requirements should be satisfied by a PEP 517 installer.
If you are using pip, you can try `pip install --use-pep517`.
********************************************************************************
!!
dist.fetch_build_eggs(dist.setup_requires)
/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
!!
********************************************************************************
Please consider removing the following classifiers in favor of a SPDX license expression:
License :: OSI Approved :: BSD License
See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
********************************************************************************
!!
self._finalize_license_expression()
running bdist_wheel
Guessing wheel URL: https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.4.post1/flash_attn-2.7.4.post1+cu11torch2.0cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
Precompiled wheel not found. Building from source...
running build
running build_py
creating build/lib.linux-x86_64-cpython-310/flash_attn
copying flash_attn/fused_softmax.py -> build/lib.linux-x86_64-cpython-310/flash_attn
copying flash_attn/flash_attn_triton.py -> build/lib.linux-x86_64-cpython-310/flash_attn
copying flash_attn/flash_attn_interface.py -> build/lib.linux-x86_64-cpython-310/flash_attn
copying flash_attn/bert_padding.py -> build/lib.linux-x86_64-cpython-310/flash_attn
copying flash_attn/flash_attn_triton_og.py -> build/lib.linux-x86_64-cpython-310/flash_attn
copying flash_attn/flash_blocksparse_attn_interface.py -> build/lib.linux-x86_64-cpython-310/flash_attn
copying flash_attn/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn
copying flash_attn/flash_blocksparse_attention.py -> build/lib.linux-x86_64-cpython-310/flash_attn
creating build/lib.linux-x86_64-cpython-310/hopper
copying hopper/generate_kernels.py -> build/lib.linux-x86_64-cpython-310/hopper
copying hopper/test_attn_kvcache.py -> build/lib.linux-x86_64-cpython-310/hopper
copying hopper/test_kvcache.py -> build/lib.linux-x86_64-cpython-310/hopper
copying hopper/padding.py -> build/lib.linux-x86_64-cpython-310/hopper
copying hopper/flash_attn_interface.py -> build/lib.linux-x86_64-cpython-310/hopper
copying hopper/benchmark_attn.py -> build/lib.linux-x86_64-cpython-310/hopper
copying hopper/setup.py -> build/lib.linux-x86_64-cpython-310/hopper
copying hopper/test_flash_attn.py -> build/lib.linux-x86_64-cpython-310/hopper
copying hopper/__init__.py -> build/lib.linux-x86_64-cpython-310/hopper
copying hopper/benchmark_split_kv.py -> build/lib.linux-x86_64-cpython-310/hopper
copying hopper/benchmark_flash_attention_fp8.py -> build/lib.linux-x86_64-cpython-310/hopper
copying hopper/test_util.py -> build/lib.linux-x86_64-cpython-310/hopper
creating build/lib.linux-x86_64-cpython-310/flash_attn/layers
copying flash_attn/layers/patch_embed.py -> build/lib.linux-x86_64-cpython-310/flash_attn/layers
copying flash_attn/layers/rotary.py -> build/lib.linux-x86_64-cpython-310/flash_attn/layers
copying flash_attn/layers/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/layers
creating build/lib.linux-x86_64-cpython-310/flash_attn/utils
copying flash_attn/utils/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/utils
copying flash_attn/utils/distributed.py -> build/lib.linux-x86_64-cpython-310/flash_attn/utils
copying flash_attn/utils/benchmark.py -> build/lib.linux-x86_64-cpython-310/flash_attn/utils
copying flash_attn/utils/generation.py -> build/lib.linux-x86_64-cpython-310/flash_attn/utils
copying flash_attn/utils/pretrained.py -> build/lib.linux-x86_64-cpython-310/flash_attn/utils
creating build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
copying flash_attn/flash_attn_triton_amd/fwd_decode.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
copying flash_attn/flash_attn_triton_amd/test.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
copying flash_attn/flash_attn_triton_amd/bench.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
copying flash_attn/flash_attn_triton_amd/utils.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
copying flash_attn/flash_attn_triton_amd/interface_torch.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
copying flash_attn/flash_attn_triton_amd/bwd_prefill.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
copying flash_attn/flash_attn_triton_amd/interface_fa.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
copying flash_attn/flash_attn_triton_amd/fwd_prefill.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
copying flash_attn/flash_attn_triton_amd/bwd_ref.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
copying flash_attn/flash_attn_triton_amd/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
copying flash_attn/flash_attn_triton_amd/fwd_ref.py -> build/lib.linux-x86_64-cpython-310/flash_attn/flash_attn_triton_amd
creating build/lib.linux-x86_64-cpython-310/flash_attn/losses
copying flash_attn/losses/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/losses
copying flash_attn/losses/cross_entropy.py -> build/lib.linux-x86_64-cpython-310/flash_attn/losses
creating build/lib.linux-x86_64-cpython-310/flash_attn/ops
copying flash_attn/ops/activations.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops
copying flash_attn/ops/layer_norm.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops
copying flash_attn/ops/rms_norm.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops
copying flash_attn/ops/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops
copying flash_attn/ops/fused_dense.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops
creating build/lib.linux-x86_64-cpython-310/flash_attn/models
copying flash_attn/models/falcon.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
copying flash_attn/models/bert.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
copying flash_attn/models/opt.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
copying flash_attn/models/vit.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
copying flash_attn/models/bigcode.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
copying flash_attn/models/gpt_neox.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
copying flash_attn/models/gpt.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
copying flash_attn/models/baichuan.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
copying flash_attn/models/llama.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
copying flash_attn/models/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
copying flash_attn/models/gptj.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
copying flash_attn/models/btlm.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
creating build/lib.linux-x86_64-cpython-310/flash_attn/modules
copying flash_attn/modules/block.py -> build/lib.linux-x86_64-cpython-310/flash_attn/modules
copying flash_attn/modules/embedding.py -> build/lib.linux-x86_64-cpython-310/flash_attn/modules
copying flash_attn/modules/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/modules
copying flash_attn/modules/mha.py -> build/lib.linux-x86_64-cpython-310/flash_attn/modules
copying flash_attn/modules/mlp.py -> build/lib.linux-x86_64-cpython-310/flash_attn/modules
creating build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
copying flash_attn/ops/triton/layer_norm.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
copying flash_attn/ops/triton/linear.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
copying flash_attn/ops/triton/k_activations.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
copying flash_attn/ops/triton/rotary.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
copying flash_attn/ops/triton/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
copying flash_attn/ops/triton/mlp.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
copying flash_attn/ops/triton/cross_entropy.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
running build_ext
building 'flash_attn_2_cuda' extension
creating /tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/build/temp.linux-x86_64-cpython-310/csrc/flash_attn
creating /tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/src
Emitting ninja build file /tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/build/temp.linux-x86_64-cpython-310/build.ninja...
Compiling objects...
Using envvar MAX_JOBS (1) as the number of workers...
[1/85] c++ -MMD -MF /tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/flash_api.o.d -pthread -B /home/zl_swuer/anaconda3/envs/llava/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/zl_swuer/anaconda3/envs/llava/include -fPIC -O2 -isystem /home/zl_swuer/anaconda3/envs/llava/include -fPIC -I/tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn -I/tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn/src -I/tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/cutlass/include -I/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include -I/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/TH -I/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/THC -I/usr/local/cuda-11.7/include -I/home/zl_swuer/anaconda3/envs/llava/include/python3.10 -c -c /tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn/flash_api.cpp -o /tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/flash_api.o -O3 -std=c++17 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=flash_attn_2_cuda -D_GLIBCXX_USE_CXX11_ABI=0
FAILED: /tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/flash_api.o
c++ -MMD -MF /tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/flash_api.o.d -pthread -B /home/zl_swuer/anaconda3/envs/llava/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/zl_swuer/anaconda3/envs/llava/include -fPIC -O2 -isystem /home/zl_swuer/anaconda3/envs/llava/include -fPIC -I/tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn -I/tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn/src -I/tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/cutlass/include -I/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include -I/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/TH -I/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/THC -I/usr/local/cuda-11.7/include -I/home/zl_swuer/anaconda3/envs/llava/include/python3.10 -c -c /tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn/flash_api.cpp -o /tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/build/temp.linux-x86_64-cpython-310/csrc/flash_attn/flash_api.o -O3 -std=c++17 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=flash_attn_2_cuda -D_GLIBCXX_USE_CXX11_ABI=0
/tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn/flash_api.cpp: In function ‘std::vector<at::Tensor> flash::mha_fwd(at::Tensor&, const at::Tensor&, const at::Tensor&, std::optional<at::Tensor>&, std::optional<at::Tensor>&, float, float, bool, int, int, float, bool, std::optional<at::Generator>)’:
/tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn/flash_api.cpp:489:13: error: invalid initialization of reference of type ‘const c10::optional<at::Generator>&’ from expression of type ‘std::optional<at::Generator>’
489 | gen_, at::cuda::detail::getDefaultCUDAGenerator());
| ^~~~
In file included from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/ATen/core/DeprecatedTypeProperties.h:9,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:32,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/ATen/core/Tensor.h:3,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/torch/csrc/utils/variadic.h:3,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/detail/static.h:3,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/python.h:3,
from /tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn/flash_api.cpp:6:
/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/ATen/core/Generator.h:160:75: note: in passing argument 1 of ‘T* at::get_generator_or_default(const c10::optional<at::Generator>&, const at::Generator&) [with T = at::CUDAGeneratorImpl]’
160 | static inline T* get_generator_or_default(const c10::optional<Generator>& gen, const Generator& default_gen) {
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
/tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn/flash_api.cpp: In function ‘std::vector<at::Tensor> flash::mha_varlen_fwd(at::Tensor&, const at::Tensor&, const at::Tensor&, std::optional<at::Tensor>&, const at::Tensor&, const at::Tensor&, std::optional<at::Tensor>&, std::optional<const at::Tensor>&, std::optional<at::Tensor>&, std::optional<at::Tensor>&, int, int, float, float, bool, bool, int, int, float, bool, std::optional<at::Generator>)’:
/tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn/flash_api.cpp:729:13: error: invalid initialization of reference of type ‘const c10::optional<at::Generator>&’ from expression of type ‘std::optional<at::Generator>’
729 | gen_, at::cuda::detail::getDefaultCUDAGenerator());
| ^~~~
In file included from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/ATen/core/DeprecatedTypeProperties.h:9,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:32,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/ATen/core/Tensor.h:3,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/torch/csrc/utils/variadic.h:3,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/detail/static.h:3,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/python.h:3,
from /tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn/flash_api.cpp:6:
/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/ATen/core/Generator.h:160:75: note: in passing argument 1 of ‘T* at::get_generator_or_default(const c10::optional<at::Generator>&, const at::Generator&) [with T = at::CUDAGeneratorImpl]’
160 | static inline T* get_generator_or_default(const c10::optional<Generator>& gen, const Generator& default_gen) {
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
/tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn/flash_api.cpp: In function ‘std::vector<at::Tensor> flash::mha_bwd(const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, std::optional<at::Tensor>&, std::optional<at::Tensor>&, std::optional<at::Tensor>&, std::optional<at::Tensor>&, float, float, bool, int, int, float, bool, std::optional<at::Generator>, std::optional<at::Tensor>&)’:
/tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn/flash_api.cpp:937:9: error: invalid initialization of reference of type ‘const c10::optional<at::Generator>&’ from expression of type ‘std::optional<at::Generator>’
937 | gen_, at::cuda::detail::getDefaultCUDAGenerator());
| ^~~~
In file included from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/ATen/core/DeprecatedTypeProperties.h:9,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:32,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/ATen/core/Tensor.h:3,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/torch/csrc/utils/variadic.h:3,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/detail/static.h:3,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/python.h:3,
from /tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn/flash_api.cpp:6:
/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/ATen/core/Generator.h:160:75: note: in passing argument 1 of ‘T* at::get_generator_or_default(const c10::optional<at::Generator>&, const at::Generator&) [with T = at::CUDAGeneratorImpl]’
160 | static inline T* get_generator_or_default(const c10::optional<Generator>& gen, const Generator& default_gen) {
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
/tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn/flash_api.cpp: In function ‘std::vector<at::Tensor> flash::mha_varlen_bwd(const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, std::optional<at::Tensor>&, std::optional<at::Tensor>&, std::optional<at::Tensor>&, const at::Tensor&, const at::Tensor&, std::optional<at::Tensor>&, int, int, float, float, bool, bool, int, int, float, bool, std::optional<at::Generator>, std::optional<at::Tensor>&)’:
/tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn/flash_api.cpp:1166:9: error: invalid initialization of reference of type ‘const c10::optional<at::Generator>&’ from expression of type ‘std::optional<at::Generator>’
1166 | gen_, at::cuda::detail::getDefaultCUDAGenerator());
| ^~~~
In file included from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/ATen/core/DeprecatedTypeProperties.h:9,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:32,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/ATen/core/Tensor.h:3,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/torch/csrc/utils/variadic.h:3,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/detail/static.h:3,
from /home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/python.h:3,
from /tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/csrc/flash_attn/flash_api.cpp:6:
/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/include/ATen/core/Generator.h:160:75: note: in passing argument 1 of ‘T* at::get_generator_or_default(const c10::optional<at::Generator>&, const at::Generator&) [with T = at::CUDAGeneratorImpl]’
160 | static inline T* get_generator_or_default(const c10::optional<Generator>& gen, const Generator& default_gen) {
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/setup.py", line 486, in run
urllib.request.urlretrieve(wheel_url, wheel_filename)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/urllib/request.py", line 241, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/urllib/request.py", line 216, in urlopen
return opener.open(url, data, timeout)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/urllib/request.py", line 525, in open
response = meth(req, response)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/urllib/request.py", line 634, in http_response
response = self.parent.error(
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/urllib/request.py", line 563, in error
return self._call_chain(*args)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/urllib/request.py", line 496, in _call_chain
result = func(*args)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/urllib/request.py", line 643, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
subprocess.run(
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v', '-j', '1']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 35, in <module>
File "/tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/setup.py", line 526, in <module>
setup(
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/__init__.py", line 117, in setup
return distutils.core.setup(**attrs)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 186, in setup
return run_commands(dist)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
dist.run_commands()
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
self.run_command(cmd)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/dist.py", line 1104, in run_command
super().run_command(command)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
cmd_obj.run()
File "/tmp/pip-install-28j6lr6z/flash-attn_e66b5f3aad0d437b87660e7979fe81be/setup.py", line 503, in run
super().run()
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/command/bdist_wheel.py", line 370, in run
self.run_command("build")
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
self.distribution.run_command(command)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/dist.py", line 1104, in run_command
super().run_command(command)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
cmd_obj.run()
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
self.distribution.run_command(command)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/dist.py", line 1104, in run_command
super().run_command(command)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
cmd_obj.run()
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 99, in run
_build_ext.run(self)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 368, in run
self.build_extensions()
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
build_ext.build_extensions(self)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 484, in build_extensions
self._build_extensions_serial()
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 510, in _build_extensions_serial
self.build_extension(ext)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 264, in build_extension
_build_ext.build_extension(self, ext)
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 565, in build_extension
objects = self.compiler.compile(
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 658, in unix_wrap_ninja_compile
_write_ninja_file_and_compile_objects(
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1574, in _write_ninja_file_and_compile_objects
_run_ninja_build(
File "/home/zl_swuer/anaconda3/envs/llava/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for flash-attn
Running setup.py clean for flash-attn
Failed to build flash-attn
ERROR: Failed to build installable wheels for some pyproject.toml based projects (flash-attn)