Skip to content

Can't delete scan or scan CRD #1101

@Spritekin

Description

@Spritekin

🐞 Bug report

Describe the bug

I was testing my setup using IAM roles. I had a few problems setting up a scan due to permissions on S3. I fixed them but then I want to delete a scan object I created and the delete gets stuck.
I run:

 % kubectl get scan
NAME                  TYPE            STATE   FINDINGS
zap-full-scan-admin   zap-full-scan           

 % kubectl delete scan zap-full-scan-admin
scan.execution.securecodebox.io "zap-full-scan-admin" deleted
... then it gets stuck here for one hour until my AWS session expires then:
error: You must be logged in to the server (Unauthorized)
exit 1

I tried uninstalling the zap scanner and operator using helm, both uninstalled fine, but the zap scan was still there and still wouldn't delete.
So I went to the uninstallation steps page ( https://docs.securecodebox.io/docs/getting-started/uninstallation ) and deleted lurkers (there were no parsers), roles, binding and service accounts. All went good.

However when I tried to delete the CRDs it got stuck in the scheduled scans:

% kubectl delete crd cascadingrules.cascading.securecodebox.io \
parsedefinitions.execution.securecodebox.io \
scancompletionhooks.execution.securecodebox.io \
scans.execution.securecodebox.io \
scantypes.execution.securecodebox.io \
scheduledscans.execution.securecodebox.io
customresourcedefinition.apiextensions.k8s.io "cascadingrules.cascading.securecodebox.io" deleted
customresourcedefinition.apiextensions.k8s.io "parsedefinitions.execution.securecodebox.io" deleted
customresourcedefinition.apiextensions.k8s.io "scancompletionhooks.execution.securecodebox.io" deleted
customresourcedefinition.apiextensions.k8s.io "scans.execution.securecodebox.io" deleted
customresourcedefinition.apiextensions.k8s.io "scantypes.execution.securecodebox.io" deleted
customresourcedefinition.apiextensions.k8s.io "scheduledscans.execution.securecodebox.io" deleted
... then if got stuck here and won't finish so I ctrl+C, but notice the scans.execution.securecodebox.io which says deleted.

% kubectl get crds                                   
NAME                                         CREATED AT
alertmanagerconfigs.monitoring.coreos.com    2021-08-04T01:47:22Z
alertmanagers.monitoring.coreos.com          2021-08-04T01:47:25Z
eniconfigs.crd.k8s.amazonaws.com             2021-06-09T02:15:07Z
podmonitors.monitoring.coreos.com            2021-08-04T01:47:27Z
probes.monitoring.coreos.com                 2021-08-04T01:47:30Z
prometheuses.monitoring.coreos.com           2021-08-04T01:47:33Z
prometheusrules.monitoring.coreos.com        2021-08-04T01:47:35Z
provisioners.karpenter.sh                    2021-12-14T05:33:10Z
scans.execution.securecodebox.io             2022-04-07T06:54:40Z    <<<<<<<<<< Won't delete
securitygrouppolicies.vpcresources.k8s.aws   2021-06-09T02:15:11Z
servicemonitors.monitoring.coreos.com        2021-08-04T01:47:37Z
thanosrulers.monitoring.coreos.com           2021-08-04T01:47:39Z

Now, I described the scan object and found it is marked for deletion:

% kubectl describe scan zap-full-scan-admin
Name:         zap-full-scan-admin
Namespace:    default
Labels:       organization=OWASP
Annotations:  <none>
API Version:  execution.securecodebox.io/v1
Kind:         Scan
Metadata:
  Creation Timestamp:             2022-04-08T00:11:04Z
  Deletion Grace Period Seconds:  0
  Deletion Timestamp:             2022-04-08T00:13:07Z     <<<<< so it is marked for deletion so should be deleted but it isn't
  Finalizers:
    s3.storage.securecodebox.io
...

I decided to test and create another scan to compare with the first scan object and got:

% kubectl apply -f zap-full-scan-admin2.yaml
Error from server (MethodNotAllowed): error when creating "zap-full-scan-admin.yaml": create not allowed while custom resource definition is terminating

So I got the scan CRD definition:

% kubectl describe crd scans.execution.securecodebox.io
Name:         scans.execution.securecodebox.io
Namespace:    
Labels:       <none>
Annotations:  controller-gen.kubebuilder.io/version: v0.4.1
API Version:  apiextensions.k8s.io/v1
Kind:         CustomResourceDefinition
Metadata:
  Creation Timestamp:             2022-04-07T06:54:40Z
  Deletion Grace Period Seconds:  0
  Deletion Timestamp:             2022-04-12T00:29:03Z
  Finalizers:
    customresourcecleanup.apiextensions.k8s.io
... a loong definition
Status:
  Accepted Names:
    Kind:       Scan
    List Kind:  ScanList
    Plural:     scans
    Singular:   scan
  Conditions:
    Last Transition Time:  2022-04-07T06:54:40Z
    Message:               no conflicts found
    Reason:                NoConflicts
    Status:                True
    Type:                  NamesAccepted
    Last Transition Time:  2022-04-07T06:54:40Z
    Message:               the initial names have been accepted
    Reason:                InitialNamesAccepted
    Status:                True
    Type:                  Established
    Last Transition Time:  2022-04-12T00:29:03Z
    Message:               CustomResource deletion is in progress
    Reason:                InstanceDeletionInProgress
    Status:                True
    Type:                  Terminating         <<<<< So it is deleting... but its stuck for some reason
  Stored Versions:
    v1
Events:  <none>

In conclusion, I got a scan CRD and a scan object that won't delete and are stuck for some unknown reason.

Steps To Reproduce

Described the problem above.

Expected behavior

I expect CRDs and Scans to delete correctly and be removed from kubernetes resources.

System (please complete the following information):

Screenshots / Logs

Additional context

Metadata

Metadata

Assignees

Labels

bugBugsdocumentationImprovements or additions to documentationgood first issueGood for newcomers

Type

No type
No fields configured for issues without a type.

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions