Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MCLEAN-124] Leverage Files.delete(Path) API to provide more accurate reason in case of failure #60

Merged
merged 6 commits into from
Nov 9, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions src/main/java/org/apache/maven/plugins/clean/Cleaner.java
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,8 @@ private boolean isSymbolicLink(Path path) throws IOException {
* @throws IOException If a file/directory could not be deleted and <code>failOnError</code> is <code>true</code>.
*/
private int delete(File file, boolean failOnError, boolean retryOnError) throws IOException {
if (!file.delete()) {
IOException error = delete(file);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not a Java error; please rename the variable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do.
To justify why I named it like this: it was to be consistent with the context of the existing terminology failOnError - it is the error in that context.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I renamed it to failure.

if (error != null) {
boolean deleted = false;

if (retryOnError) {
Expand All @@ -289,18 +290,18 @@ private int delete(File file, boolean failOnError, boolean retryOnError) throws
} catch (InterruptedException e) {
// ignore
}
deleted = file.delete() || !file.exists();
deleted = delete(file) == null || !file.exists();
}
} else {
deleted = !file.exists();
}

if (!deleted) {
if (failOnError) {
throw new IOException("Failed to delete " + file);
throw new IOException("Failed to delete " + file, error);
} else {
if (log.isWarnEnabled()) {
log.warn("Failed to delete " + file);
log.warn("Failed to delete " + file, error);
}
return 1;
}
Expand All @@ -310,6 +311,15 @@ private int delete(File file, boolean failOnError, boolean retryOnError) throws
return 0;
}

private static IOException delete(File file) {
try {
Files.delete(file.toPath());
} catch (IOException e) {
return e;
}
return null;
}

private static class Result {

private int failures;
Expand Down
183 changes: 183 additions & 0 deletions src/test/java/org/apache/maven/plugins/clean/CleanerTest.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.maven.plugins.clean;

import java.io.IOException;
import java.nio.file.AccessDeniedException;
import java.nio.file.DirectoryNotEmptyException;
import java.nio.file.Path;
import java.nio.file.attribute.PosixFilePermission;
import java.nio.file.attribute.PosixFilePermissions;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Set;

import org.apache.maven.plugin.logging.Log;
import org.apache.maven.plugin.testing.SilentLog;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;

import static java.nio.file.Files.createDirectory;
import static java.nio.file.Files.createFile;
import static java.nio.file.Files.exists;
import static java.nio.file.Files.getPosixFilePermissions;
import static java.nio.file.Files.setPosixFilePermissions;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertInstanceOf;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;

class CleanerTest {

private boolean warnEnabled;

/**
* Use a {@code LinkedHashMap} to preserve the order of the logged warnings.
*/
private final Map<CharSequence, Throwable> warnings = new LinkedHashMap<>();

/**
* Ideally we should use a mocking framework such as Mockito for this, but alas, this project has no such dependency.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can simply add Mockito to dependencies

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, good to know. I had to settle for Mockito 4.x/Java 8 instead of 5.x/Java 11, but it's an improvement nonetheless.
I also needed to fix the tests for Windows paths.

*/
private final Log log = new SilentLog() {

@Override
public boolean isWarnEnabled() {
return warnEnabled;
}

@Override
public void warn(CharSequence content, Throwable error) {
warnings.put(content, error);
}
};

@Test
void deleteSucceedsDeeply(@TempDir Path tempDir) throws Exception {
final Path basedir = createDirectory(tempDir.resolve("target"));
final Path file = createFile(basedir.resolve("file"));
final Cleaner cleaner = new Cleaner(null, log, false, null, null);
cleaner.delete(basedir.toFile(), null, false, true, false);
assertFalse(exists(basedir));
assertFalse(exists(file));
}

@Test
void deleteFailsWithoutRetryWhenNoPermission(@TempDir Path tempDir) throws Exception {
warnEnabled = true;
final Path basedir = createDirectory(tempDir.resolve("target"));
createFile(basedir.resolve("file"));
final Set<PosixFilePermission> initialPermissions = getPosixFilePermissions(basedir);
final String rwxrwxr_x = PosixFilePermissions.toString(initialPermissions);
// Prevent directory listing, which will result in a DirectoryNotEmptyException.
final String rw_rw_r__ = rwxrwxr_x.replace('x', '-');
final Set<PosixFilePermission> permissions = PosixFilePermissions.fromString(rw_rw_r__);
setPosixFilePermissions(basedir, permissions);
try {
final Cleaner cleaner = new Cleaner(null, log, false, null, null);
final IOException exception =
assertThrows(IOException.class, () -> cleaner.delete(basedir.toFile(), null, false, true, false));
assertTrue(warnings.isEmpty());
assertEquals("Failed to delete " + basedir, exception.getMessage());
final DirectoryNotEmptyException cause =
assertInstanceOf(DirectoryNotEmptyException.class, exception.getCause());
assertEquals(basedir.toString(), cause.getMessage());
} finally {
// Allow the tempDir to be cleared by the @TempDir extension.
setPosixFilePermissions(basedir, initialPermissions);
}
}

@Test
void deleteFailsAfterRetryWhenNoPermission(@TempDir Path tempDir) throws Exception {
final Path basedir = createDirectory(tempDir.resolve("target"));
createFile(basedir.resolve("file"));
final Set<PosixFilePermission> initialPermissions = getPosixFilePermissions(basedir);
final String rwxrwxr_x = PosixFilePermissions.toString(initialPermissions);
// Prevent directory listing, which will result in a DirectoryNotEmptyException.
final String rw_rw_r__ = rwxrwxr_x.replace('x', '-');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is replace needed here? Could this just be a string literal?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace is not strictly needed, it could be a string literal, but the replace reflects better the intention. I just want to remove the executable flag, and leave any other "rw" settings as they were. That's exactly what this replace expresses. If a make it a string literal, it is not clear to the reader that in fact I only mean to change the executable flag. The name of the variable rw_rw_r__ is merely suggestive, to reflect the sort of value the string can have.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels too clever by half. Let's just make it a string literal.

final Set<PosixFilePermission> permissions = PosixFilePermissions.fromString(rw_rw_r__);
setPosixFilePermissions(basedir, permissions);
try {
final Cleaner cleaner = new Cleaner(null, log, false, null, null);
final IOException exception =
assertThrows(IOException.class, () -> cleaner.delete(basedir.toFile(), null, false, true, true));
assertEquals("Failed to delete " + basedir, exception.getMessage());
final DirectoryNotEmptyException cause =
assertInstanceOf(DirectoryNotEmptyException.class, exception.getCause());
assertEquals(basedir.toString(), cause.getMessage());
} finally {
// Allow the tempDir to be cleared by the @TempDir extension.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be what I'm missing. The annotation doesn't work with the permissions you set?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it does, maybe it doesn't. I didn't check. I deem it good practice to reset any modified persistent state so that I don't have to make assumptions about how "powerful" the @TempDir extension really is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked, the @TempDir framework is smart enough to fix the permissions to that the tmp dir can be deleted. So I removed all the try-catches.

setPosixFilePermissions(basedir, initialPermissions);
}
}

@Test
void deleteLogsWarningWithoutRetryWhenNoPermission(@TempDir Path tempDir) throws Exception {
warnEnabled = true;
final Path basedir = createDirectory(tempDir.resolve("target"));
final Path file = createFile(basedir.resolve("file"));
final Set<PosixFilePermission> initialPermissions = getPosixFilePermissions(basedir);
final String rwxrwxr_x = PosixFilePermissions.toString(initialPermissions);
final String r_xr_xr_x = rwxrwxr_x.replace('w', '-');
final Set<PosixFilePermission> permissions = PosixFilePermissions.fromString(r_xr_xr_x);
setPosixFilePermissions(basedir, permissions);
try {
final Cleaner cleaner = new Cleaner(null, log, false, null, null);
assertDoesNotThrow(() -> cleaner.delete(basedir.toFile(), null, false, false, false));
assertEquals(2, warnings.size());
final Iterator<Entry<CharSequence, Throwable>> it =
warnings.entrySet().iterator();
final Entry<CharSequence, Throwable> warning1 = it.next();
assertEquals("Failed to delete " + file, warning1.getKey());
final AccessDeniedException cause1 = assertInstanceOf(AccessDeniedException.class, warning1.getValue());
assertEquals(file.toString(), cause1.getMessage());
final Entry<CharSequence, Throwable> warning2 = it.next();
assertEquals("Failed to delete " + basedir, warning2.getKey());
final DirectoryNotEmptyException cause2 =
assertInstanceOf(DirectoryNotEmptyException.class, warning2.getValue());
assertEquals(basedir.toString(), cause2.getMessage());
} finally {
setPosixFilePermissions(basedir, initialPermissions);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you need to reset this here? Feels like the object shouldn't be used again

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid resource exhaustion. It's good practice to reset persistent state which is modified in a test. The files I make in this test are persistent, and I rely on the @TempDir extension in the JUnit framework to delete them. If I leave permissions as they are, i.e. deliberately restricted so that files cannot be deleted, I imagine it could be that deleting the directory fails, resulting not per se in an exception but in lingering temporary directories. Over time, lingering temporary directories can cause "no space left on device" on test systems. By resetting the permissions, I reset the persistent state, allowing the JUnit framework to clear the temporary directory, avoiding resource exhaustion.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something's off with that. Test resources including files should not be shared between test methods. Per Junit docs "The temporary directory will be shared by all tests in a class when the annotation is present on a static field or on a parameter of a @BeforeAll method. Otherwise — for example, when @tempdir is only used on instance fields or on parameters in test, @BeforeEach, or @AfterEach methods — each test will use its own temporary directory."

Copy link
Contributor Author

@peterdemaeyer peterdemaeyer Nov 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is a misunderstanding. I agree that test resources including files should not be shared between test methods. That's exactly why I did not share them between tests. According to you quote of the JUnit docs, they fall in the category "@TempDir is used on parameters in tests", in which case "each test will use its own temporary directory", which is exactly what we want, no?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I don't think these are what I would call "persistent'. Either way, let's start with the assumption you don't need this. Take out the finally block and see if everything still passes.

}
}

@Test
void deleteDoesNotLogAnythingWhenNoPermissionAndWarnDisabled(@TempDir Path tempDir) throws Exception {
warnEnabled = false;
final Path basedir = createDirectory(tempDir.resolve("target"));
createFile(basedir.resolve("file"));
final Set<PosixFilePermission> initialPermissions = getPosixFilePermissions(basedir);
final String rwxrwxr_x = PosixFilePermissions.toString(initialPermissions);
final String r_xr_xr_x = rwxrwxr_x.replace('w', '-');
final Set<PosixFilePermission> permissions = PosixFilePermissions.fromString(r_xr_xr_x);
setPosixFilePermissions(basedir, permissions);
try {
final Cleaner cleaner = new Cleaner(null, log, false, null, null);
assertDoesNotThrow(() -> cleaner.delete(basedir.toFile(), null, false, false, false));
assertTrue(warnings.isEmpty());
} finally {
setPosixFilePermissions(basedir, initialPermissions);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why reset?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my other answer.

}
}
}
Loading