Managing Large Clusters with kubectl

In clusters with thousands of resources, careless kubectl usage can overload the API server and cause slow responses. Pagination, chunking, and selective queries are essential techniques.

Using the --limit Flag

The --limit flag enables server-side pagination, fetching resources in manageable chunks:

# Fetch pods in chunks of 100
kubectl get pods -A --limit=100

# Paginate through deployments
kubectl get deployments -A --limit=50

# Combine with output format
kubectl get pods -A --limit=200 -o wide

kubectl automatically follows the continue token to retrieve all pages, but each page is a smaller API call.

Avoiding List-All Patterns

Never list all resources across all namespaces without filters in large clusters:

# Bad: fetches everything
kubectl get pods -A

# Good: scope to a namespace
kubectl get pods -n production

# Good: use label selectors
kubectl get pods -l app=myapp -A

# Good: use field selectors for specific states
kubectl get pods --field-selector=status.phase!=Running -A

Efficient Querying

# Get only names (minimal data transfer)
kubectl get pods -o name -n production

# Use jsonpath to extract specific fields
kubectl get pods -o jsonpath='{.items[*].metadata.name}' -n production

# Count resources without fetching full objects
kubectl get pods -n production --no-headers | wc -l

# Use server-side apply for large manifests
kubectl apply --server-side -f large-manifest.yaml

Chunked List Requests

For programmatic access, use the chunk size parameter:

# Set chunk size via config
kubectl get pods -A --chunk-size=500

# Monitor API request sizes with verbosity
kubectl get pods -A -v=6 2>&1 | grep "GET"

As a rule, always include a namespace or label selector. Avoid -A (all namespaces) in automation scripts, and prefer --watch over repeated list calls for monitoring.