Disclaimer: I'm not totally complete with that description, because the problem is bigger,
but lets start with that and have a proper mature discussion.
I basically expected behavior that matches other adapters provided in the package (namely Azure), and kept the same option names.
Here are 3 problems, (all are solved with the PR linked).
All of which consider the following setup:
Consider
- an azure client initiated with valid credentials
- a blob container that has
- 1 file under path
a/
- 5 files under path
b/
- 1 file under path
c/
In node shell:
> client = require('pkgcloud').storage.createClient({provider: 'azure', process.env.AZURE_ACCOUNT, process.env.AZURE_API_KEY});1
1
> container = process.env.AZURE_CONTAINER;1
1
Problem one - limit
Scenario
When I list files and provide options.limit and options.prefix
> client.getFiles(container, {limit:2}, (e,f) => console.log('Result', e || f.length));1
1
Expected
I expect only as much files as the provided limit
Found
I get all the files
Problem 2 - paging marker
According to Azure,
A paged list should return an opaque cursor identifier - aka marker - by which paging may continue:
marker - Optional. A string value that identifies the portion of the list to be returned with the next list operation. The operation returns a marker value within the response body if the list returned was not complete. The marker value may then be used in a subsequent call to request the next set of list items.
The marker value is opaque to the client.
See: list-blobs#uri-parameters
While Rackspace can use the file.name as a marker - Azure cannot, while the present API does not allow the marker to float back to the user.
Problem 3 - selector by a prefix
Scenario
When I provide a prefix
> client.getFiles(container, {prefix: 'a/'}, (e,f) => console.log('Result', e || f.length));1
1
Expected
I expect only files under the provided prefix
Found
I get all files
NOTE: If the bucket contains more than 10K files - there is no way to list files beyond the 10Kth file.
Disclaimer: I'm not totally complete with that description, because the problem is bigger,
but lets start with that and have a proper mature discussion.
I basically expected behavior that matches other adapters provided in the package (namely Azure), and kept the same option names.
Here are 3 problems, (all are solved with the PR linked).
All of which consider the following setup:
Consider
a/b/c/In node shell:
Problem one - limit
Scenario
When I list files and provide
options.limitandoptions.prefixExpected
I expect only as much files as the provided limit
Found
I get all the files
Problem 2 - paging marker
According to Azure,
A paged list should return an opaque cursor identifier - aka marker - by which paging may continue:
See: list-blobs#uri-parameters
While Rackspace can use the
file.nameas a marker - Azure cannot, while the present API does not allow the marker to float back to the user.Problem 3 - selector by a prefix
Scenario
When I provide a prefix
Expected
I expect only files under the provided prefix
Found
I get all files
NOTE: If the bucket contains more than 10K files - there is no way to list files beyond the 10Kth file.