[d-kernel] [PATCH] amdkfd: partially revert 96c211f1f9ef82183493f4ceed4e347b52849149

[d-kernel] [PATCH] amdkfd: partially revert 96c211f1f9ef82183493f4ceed4e347b52849149

[lakostis]

From: "Konstantin A. Lepikhov" <lakostis@altlinux.ru>

This fixes the issue with hanging/crashing blender/opencl applications
See https://github.com/ROCm/ROCm/issues/2596#issuecomment-1874371163
(Can be applied to v6.6.9 kernel)
---
 drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c | 26 ++++++++++----------
 1 file changed, 13 insertions(+), 13 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c b/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c
index 62b205dac63a..efb05acea2f8 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c
@@ -330,12 +330,6 @@ static void kfd_init_apertures_vi(struct kfd_process_device *pdd, uint8_t id)
 	pdd->gpuvm_limit =
 		pdd->dev->kfd->shared_resources.gpuvm_size - 1;
 
-	/* dGPUs: the reserved space for kernel
-	 * before SVM
-	 */
-	pdd->qpd.cwsr_base = SVM_CWSR_BASE;
-	pdd->qpd.ib_base = SVM_IB_BASE;
-
 	pdd->scratch_base = MAKE_SCRATCH_APP_BASE_VI();
 	pdd->scratch_limit = MAKE_SCRATCH_APP_LIMIT(pdd->scratch_base);
 }
@@ -345,18 +339,18 @@ static void kfd_init_apertures_v9(struct kfd_process_device *pdd, uint8_t id)
 	pdd->lds_base = MAKE_LDS_APP_BASE_V9();
 	pdd->lds_limit = MAKE_LDS_APP_LIMIT(pdd->lds_base);
 
-	pdd->gpuvm_base = PAGE_SIZE;
+	/* Raven needs SVM to support graphic handle, etc. Leave the small
+	 * reserved space before SVM on Raven as well, even though we don't
+	 * have to.
+	 * Set gpuvm_base and gpuvm_limit to CANONICAL addresses so that they
+	 * are used in Thunk to reserve SVM.
+	 */
+	pdd->gpuvm_base = SVM_USER_BASE;
 	pdd->gpuvm_limit =
 		pdd->dev->kfd->shared_resources.gpuvm_size - 1;
 
 	pdd->scratch_base = MAKE_SCRATCH_APP_BASE_V9();
 	pdd->scratch_limit = MAKE_SCRATCH_APP_LIMIT(pdd->scratch_base);
-
-	/*
-	 * Place TBA/TMA on opposite side of VM hole to prevent
-	 * stray faults from triggering SVM on these pages.
-	 */
-	pdd->qpd.cwsr_base = pdd->dev->kfd->shared_resources.gpuvm_size;
 }
 
 int kfd_init_apertures(struct kfd_process *process)
@@ -413,6 +407,12 @@ int kfd_init_apertures(struct kfd_process *process)
 					return -EINVAL;
 				}
 			}
+
+			/* dGPUs: the reserved space for kernel
+			 * before SVM
+			 */
+			pdd->qpd.cwsr_base = SVM_CWSR_BASE;
+			pdd->qpd.ib_base = SVM_IB_BASE;
 		}
 
 		dev_dbg(kfd_device, "node id %u\n", id);
-- 
2.42.1

Re: [d-kernel] [PATCH] amdkfd: partially revert 96c211f1f9ef82183493f4ceed4e347b52849149

[Vitaly Chikunov]

Konstantin,

On Wed, Jan 03, 2024 at 10:15:37AM +0100, lakostis@altlinux.org wrote:
> From: "Konstantin A. Lepikhov" <lakostis@altlinux.ru>
> 
> This fixes the issue with hanging/crashing blender/opencl applications
> See https://github.com/ROCm/ROCm/issues/2596#issuecomment-1874371163
> (Can be applied to v6.6.9 kernel)

Это срочно? Почему не подождать патча из апстрима - тем более он принят
в git подсистемы drm:
  https://lore.kernel.org/all/CADnq5_PF8DrWHiSfwq3ju0eKkX0dN8JwZHwnCoBpzXFhucFmzA@mail.gmail.com/ (3 Jan 2024)
  https://gitlab.freedesktop.org/agd5f/linux/-/commits/drm-next/?ref_type=heads (Jan 05, 2024)
Думаю, это займет недели 2.

> ---
>  drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c | 26 ++++++++++----------
>  1 file changed, 13 insertions(+), 13 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c b/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c
> index 62b205dac63a..efb05acea2f8 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c
> @@ -330,12 +330,6 @@ static void kfd_init_apertures_vi(struct kfd_process_device *pdd, uint8_t id)
>  	pdd->gpuvm_limit =
>  		pdd->dev->kfd->shared_resources.gpuvm_size - 1;
>  
> -	/* dGPUs: the reserved space for kernel
> -	 * before SVM
> -	 */
> -	pdd->qpd.cwsr_base = SVM_CWSR_BASE;
> -	pdd->qpd.ib_base = SVM_IB_BASE;
> -
>  	pdd->scratch_base = MAKE_SCRATCH_APP_BASE_VI();
>  	pdd->scratch_limit = MAKE_SCRATCH_APP_LIMIT(pdd->scratch_base);
>  }
> @@ -345,18 +339,18 @@ static void kfd_init_apertures_v9(struct kfd_process_device *pdd, uint8_t id)
>  	pdd->lds_base = MAKE_LDS_APP_BASE_V9();
>  	pdd->lds_limit = MAKE_LDS_APP_LIMIT(pdd->lds_base);
>  
> -	pdd->gpuvm_base = PAGE_SIZE;
> +	/* Raven needs SVM to support graphic handle, etc. Leave the small
> +	 * reserved space before SVM on Raven as well, even though we don't
> +	 * have to.
> +	 * Set gpuvm_base and gpuvm_limit to CANONICAL addresses so that they
> +	 * are used in Thunk to reserve SVM.
> +	 */
> +	pdd->gpuvm_base = SVM_USER_BASE;
>  	pdd->gpuvm_limit =
>  		pdd->dev->kfd->shared_resources.gpuvm_size - 1;
>  
>  	pdd->scratch_base = MAKE_SCRATCH_APP_BASE_V9();
>  	pdd->scratch_limit = MAKE_SCRATCH_APP_LIMIT(pdd->scratch_base);
> -
> -	/*
> -	 * Place TBA/TMA on opposite side of VM hole to prevent
> -	 * stray faults from triggering SVM on these pages.
> -	 */
> -	pdd->qpd.cwsr_base = pdd->dev->kfd->shared_resources.gpuvm_size;
>  }
>  
>  int kfd_init_apertures(struct kfd_process *process)
> @@ -413,6 +407,12 @@ int kfd_init_apertures(struct kfd_process *process)
>  					return -EINVAL;
>  				}
>  			}
> +
> +			/* dGPUs: the reserved space for kernel
> +			 * before SVM
> +			 */
> +			pdd->qpd.cwsr_base = SVM_CWSR_BASE;
> +			pdd->qpd.ib_base = SVM_IB_BASE;
>  		}
>  
>  		dev_dbg(kfd_device, "node id %u\n", id);
> -- 
> 2.42.1
> 
> _______________________________________________
> devel-kernel mailing list
> devel-kernel@lists.altlinux.org
> https://lists.altlinux.org/mailman/listinfo/devel-kernel

Re: [d-kernel] [PATCH] amdkfd: partially revert 96c211f1f9ef82183493f4ceed4e347b52849149

[Konstantin Lepikhov]

Hi Vitaly!

On 01/06/2024, at 10:27:10 AM you wrote:

> Konstantin,
> 
> On Wed, Jan 03, 2024 at 10:15:37AM +0100, lakostis@altlinux.org wrote:
> > From: "Konstantin A. Lepikhov" <lakostis@altlinux.ru>
> > 
> > This fixes the issue with hanging/crashing blender/opencl applications
> > See https://github.com/ROCm/ROCm/issues/2596#issuecomment-1874371163
> > (Can be applied to v6.6.9 kernel)
> 
> Это срочно? Почему не подождать патча из апстрима - тем более он принят
> в git подсистемы drm:
>   https://lore.kernel.org/all/CADnq5_PF8DrWHiSfwq3ju0eKkX0dN8JwZHwnCoBpzXFhucFmzA@mail.gmail.com/ (3 Jan 2024)
>   https://gitlab.freedesktop.org/agd5f/linux/-/commits/drm-next/?ref_type=heads (Jan 05, 2024)
> Думаю, это займет недели 2.
> 
Без этого патча opencl на amd/hip сломан. Если это неважно то можно и подождать.

-- 
WBR et al.