Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: configure file size limit #668

Open
wants to merge 29 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
cdb5b56
Add support to index multiple file extensions
Feb 18, 2017
5f096c4
Add test for indexing multiple file types
Feb 18, 2017
7dc4477
Fix wrong phpDoc type
Feb 18, 2017
f7175bc
Filter invalid file types and use default list as fallback
Feb 18, 2017
9433694
Let JsonMapper intialize the options
Feb 18, 2017
39cfbda
Add test for fileTypes option
Feb 18, 2017
3c33e7f
Initialize options with default values when not provided by client
Feb 18, 2017
d2e5048
Update testIndexingMultipleFileTypes
Feb 18, 2017
b9d0d1b
Add missing namespace in OptionsTest
Feb 18, 2017
9067b44
Fix wrong classname for options test
Feb 18, 2017
1e319c7
Wipe index when on configuration change
Feb 24, 2017
58c82e6
Add list of valid indexer options
Mar 2, 2017
44a942e
Implement didChangeConfiguration event
Mar 2, 2017
940eb97
Pass options and indexer to workspace
Mar 2, 2017
5b1b6bf
Add tests
Mar 2, 2017
707c97f
Merge branch 'master' of github.com:felixfbecker/php-language-server …
Mar 2, 2017
1e73d08
Improve gettting changed options
Mar 4, 2017
1f90b4e
Update options one by one to update all instance
Mar 4, 2017
ca225ff
Remove emitting wipe events
Mar 4, 2017
c4568bf
Accept different types/formats from clients
Mar 4, 2017
5308e7a
Add new tests and update old ones
Mar 4, 2017
a06057b
Fix phpcs warnings/errors
Mar 4, 2017
23a40f0
Let didChangeConfiguration decide what options are interesting for th…
Mar 4, 2017
f4f1067
Change didChangeConfiguration doc to protocol wording
Mar 4, 2017
9cc2736
Merge remote-tracking branch 'upstream/master' into feature/allow-con…
JSteitz Aug 29, 2018
09fbec2
Refactor pull request
JSteitz Aug 29, 2018
a5417cd
Fix risky test warning
JSteitz Aug 29, 2018
24e3b77
Add option to set file size limit
JSteitz Aug 29, 2018
b7bcf00
Add basic test
JSteitz Aug 29, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/Indexer.php
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,7 @@ private function indexFiles(array $files): Promise
yield timeout();
$this->client->window->logMessage(MessageType::LOG, "Parsing $uri");
try {
$document = yield $this->documentLoader->load($uri);
$document = yield $this->documentLoader->load($uri, $this->options->fileSizeLimit);
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@felixfbecker I think this is the right place to pass the file size limit, what do you think?
Just asking, because I need to adjust a lot of tests for that.

if (!isVendored($document, $this->composerJson)) {
$this->client->textDocument->publishDiagnostics($uri, $document->getDiagnostics());
}
Expand Down
41 changes: 40 additions & 1 deletion src/Options.php
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,13 @@ class Options
*/
public $fileTypes = ['.php'];

/**
* Maximum file size to index
*
* @var int
*/
public $fileSizeLimit = 150000;

/**
* Validate/Filter input and set options for file types
*
Expand All @@ -20,13 +27,29 @@ class Options
public function setFileTypes(array $fileTypes)
{
$fileTypes = filter_var_array($fileTypes, FILTER_SANITIZE_STRING);
$fileTypes = filter_var($fileTypes, FILTER_CALLBACK, ['options' => [$this, 'filterFileTypes']]);
$fileTypes = filter_var($fileTypes, FILTER_CALLBACK, ['options' => [$this, 'filterFileTypes']]);
$fileTypes = array_filter($fileTypes, 'strlen');
$fileTypes = array_values($fileTypes);

$this->fileTypes = !empty($fileTypes) ? $fileTypes : $this->fileTypes;
}

/**
* Validate/Filter input and set option for file size limit
*
* @param string $fileSizeLimit Size in human readable format or -1 for unlimited
*/
public function setFileSizeLimit(string $fileSizeLimit)
{
$fileSizeLimit = filter_var($fileSizeLimit, FILTER_SANITIZE_STRING);

if ($fileSizeLimit === '-1') {
$this->fileSizeLimit = PHP_INT_MAX;
} else {
$this->fileSizeLimit = $this->convertFileSize($fileSizeLimit);
}
}

/**
* Filter valid file type
*
Expand All @@ -47,4 +70,20 @@ private function filterFileTypes(string $fileType)

return $fileType;
}

/**
* Convert human readable file size to byte
*
* @param string $fileSize
* @return int
*/
private function convertFileSize(string $fileSize)
{
preg_match('/(\d+)(\w)/', $fileSize, $match);
$sizes = 'KMG';
$size = (int) $match[1];
$factor = strpos($sizes, strtoupper($match[2])) + 1;

return $size * pow(1000, $factor);
}
}
7 changes: 3 additions & 4 deletions src/PhpDocumentLoader.php
Original file line number Diff line number Diff line change
Expand Up @@ -100,13 +100,12 @@ public function getOrLoad(string $uri): Promise
* The document is NOT added to the list of open documents, but definitions are registered.
*
* @param string $uri
* @param int $limit
* @return Promise <PhpDocument>
*/
public function load(string $uri): Promise
public function load(string $uri, int $limit): Promise
{
return coroutine(function () use ($uri) {

$limit = 150000;
return coroutine(function () use ($uri, $limit) {
$content = yield $this->contentRetriever->retrieve($uri);
$size = strlen($content);
if ($size > $limit) {
Expand Down