Find duplicate code with CPD

A copy/paste detector

I’m sure that you know PMD, the famous static code analyzer. But have you ever tried CPD?

Installation

CPD belongs to PMD. You have to install PMD first.

[code]
$ cd $HOME
$ wget https://github.com/pmd/pmd/releases/download/pmd_releases%2F5.8.1/pmd-bin-5.8.1.zip
$ unzip pmd-bin-5.8.1.zip
$ alias pmd="$HOME/tools/pmd-bin-5.8.1/bin/run.sh pmd"
$ alias cpd="$HOME/tools/pmd-bin-5.8.1/bin/run.sh cpd"
[/code]

Command line usage

I give you some examples here.

[code]
cpd –minimum-tokens 20 –language java –files src > cpd-20.txt
cpd –minimum-tokens 20 –format csv –files src > cpd-20.csv
[/code]

minimum-tokens (minimum duplicate size) and files (source directory) are the only required options. Then you can specify the language and the report format. What I really like is the csv format. I sort the results in Excel (tokens) and attack the Top 5 duplicates.

Conclusion

CPD helps you to find copied and pasted code. Duplicate code tells you how to improve the code structure.

Links

https://pmd.github.io/pmd-5.8.1/usage/cpd-usage.html

Why are engineering values important?

Everything else will grow from values and behaviour

Wikipedia writes „… Values can be defined as broad preferences concerning appropriate courses of action or outcomes. As such, values reflect a person’s sense of right and wrong or what „ought“ to be… Values tend to influence attitudes and behavior [value].“

And culture comes from these shared values that causes everyone to work with purpose.
A healthy culture makes developers more productive and happy by removing unnecessary overhead. A great culture brings teams together or helps the programmer to innovate and raises awareness about how to create better code.

So how do we get there? Here comes a list of actions and principles:

Design & Code
hold engineering design session
present proposals for new features
not writing bad code
say „no“ if a change is bad for the software
always looking for a better solution
build code, get feedback, improve or dump it
if it’s broken – fix it
do less better
ship small pieces of valuable code and a short period of time and learn

Shipping a feature
define success metrics
build a feature and get feedback from code review
use feature flags
deploy the simplest version possible – staff only
fix Bugs, make improvments before launching it to beta customers
Beta: Fix Bugs, make more improvments before launching a feature for everyone

Customer and products
focus on the customer
everybody’s goal is to make the customer’s job easier and more effective
work on projects you love
anyone has the right to freely express the ideas regarding future development of products
we are proud of our products and quality of work

These are just a few examples that could lead to a more healthier culture. A great culture emphasizes transparency and honesty, respects autonomy and encourages collaboration.

How to specify Java Source and Target Compatibility in a Gradle multi project

I have a multiple project setup and I want to build top level projects using different JDKs.

I did some experiments by myself and the following configuration worked for me
(using gradle 3.1 and the wrapper function).

At the root level my gradle.properties file looks like this:

[code]
org.gradle.daemon=true
JDK7_HOME=/usr/lib/jvm/jdk-7-oracle-x64
JDK8_HOME=/usr/lib/jvm/oracle-java8-jdk-amd64
[/code]

Default JDK should be JDK 7. So I defined this in my root build.gradle file

[code]

subprojects {

apply plugin: ‚java‘
sourceCompatibility = 1.7
targetCompatibility = 1.7

compileJava.doFirst {
println ’source compatibility: ‚ + sourceCompatibility
println ‚targetCompatibility: ‚ + targetCompatibility
}

tasks.withType(JavaCompile) {
if (project.hasProperty(‚JDK7_HOME‘)) {
options.fork = true
options.bootClasspath = ‚$JDK7_HOME/jre/lib/rt.jar:$JDK7_HOME/jre/lib/jce.jar‘
}
}

test.doFirst {
println ‚executing tests with JDK ‚ + test.executable
}

tasks.withType(Test) {
if (project.hasProperty(‚JDK7_HOME‘)) {
test.executable = ‚$JDK7_HOME/bin/java‘
}
}

}

[/code]

With this configuration the source and target compatibility is now set to 1.7 for every
sub project. I use compileJava.doFirst and test.doFirst to verify that everything works as expected.

Now, in my sub project B I want to use Java 8. The following did the trick:
build.gradle

[code]

tasks.withType(JavaCompile) {
if (project.hasProperty(‚JDK8_HOME‘)) {
sourceCompatibility = 1.8
targetCompatibility = 1.8
options.fork = true
options.bootClasspath = ‚$JDK8_HOME/jre/lib/rt.jar:$JDK8_HOME/jre/lib/jce.jar‘;
}
}

tasks.withType(Test) {
if (project.hasProperty(‚JDK8_HOME‘)) {
test.executable = ‚$JDK8_HOME/bin/java‘
}
}

[/code]

That it’s. I hope this has helped you.

If not, please check these ones:

http://stackoverflow.com/questions/21028438/gradle-sourcecompatibility-has-no-effect-to-subprojects

http://stackoverflow.com/questions/16679593/gradle-compilejava-task-warning-options-bootstrap-class-path-not-set-in-conju

http://stackoverflow.com/questions/30571210/how-configure-multiple-gradle-properties-files-in-gradle-for-multiple-projects

 

Analyze application performance with CDI Interceptors

Using interceptors for analyzing performance issues

Do you know statements in your code like this?

[code]

long start = System.currentTimeMillis();
//some computing…
long end = System.currentTimeMillis();
long duration = end – start;

[/code]

Analyzing how long the execution of a method takes is a cross-cutting concern. Cross-cutting concerns like transactions, caching or measuring latencies are aspects of a program that cannot be cleanly decomposed from the rest of the system. So they result in either code duplication or significant dependencies between systems.
CDI Interceptors are perfect for encapsulating such concerns. You can use the interceptor concept to implement such cross-cutting concerns in an isolated and reusable manner.

For my example I use Weld, the reference implementation of CDI. CDI stands for Context and Dependency Injection. It helps you to improve the structure of your code and defines a powerful set of services like dependency injection, contextual lifecycles, events, decorators and interceptors.

Here comes my toy example. execute() is annotated with @Measured. This annotations intercepts all calls to methods annotated with this annotation of every CDI Bean that is managed by the container.

[code]
package de.claudioaltamura.labs.cdi.interceptor;

import java.util.Random;
import java.util.concurrent.TimeUnit;

public class Service {

private Random random = new Random();

@Measured
public long execute() throws InterruptedException {

//long computation
TimeUnit.SECONDS.sleep(randomTime());

return 0;
}

private long randomTime() {
return random.nextInt(10) + 1;
}

}
[/code]

The annotation itself is very simple. @InterceptorBinding specifies that the annotation is an interceptor binding type. Interceptor bindings associate interceptors with target beans.

[code]
package de.claudioaltamura.labs.cdi.interceptor;

import static java.lang.annotation.ElementType.METHOD;
import static java.lang.annotation.ElementType.TYPE;
import static java.lang.annotation.RetentionPolicy.RUNTIME;

import java.lang.annotation.Retention;
import java.lang.annotation.Target;

import javax.interceptor.InterceptorBinding;

@InterceptorBinding
@Target({METHOD, TYPE})
@Retention(RUNTIME)
public @interface Measured {
}
[/code]

Now I show you the interceptor. MeasureTimeInterceptor finally gets all calls and just logs the execution time of your method.

[code]
package de.claudioaltamura.labs.cdi.interceptor;

import java.util.concurrent.TimeUnit;

import javax.interceptor.AroundInvoke;
import javax.interceptor.Interceptor;
import javax.interceptor.InvocationContext;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@Measured
@Interceptor
public class MeasureTimeInterceptor {

@AroundInvoke
private Object intercept(InvocationContext ic) throws Exception {
long start = System.nanoTime();
Object obj = ic.proceed();
long end = System.nanoTime();
long durationInMs = TimeUnit.NANOSECONDS.toMillis(end – start);

Logger log = LoggerFactory.getLogger(ic.getMethod().getDeclaringClass());
log.info(ic.getMethod().getName() + " – " + durationInMs + " ms");
return obj;
}

}
[/code]

Last step is the CDI configuration (bean.xml).

[code]
<beans xmlns="http://xmlns.jcp.org/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee
http://xmlns.jcp.org/xml/ns/javaee/beans_1_1.xsd" bean-discovery-mode="all">

<interceptors>
<class>de.claudioaltamura.labs.cdi.interceptor.MeasureTimeInterceptor</class>
</interceptors>

</beans
[/code]

That’s it. A last point, I would like to talk about briefly is the decorator pattern. Decorators and interceptors are very similar concepts. So what are the differences?

"... Decorators and Interceptors are similar because they can be both used to "enrich" or "decorate" one method call with additional tasks. However Interceptors are used for general tasks, which are not related with your class in particular (e.g. auditing). On the other hand, Decorators, do bear with them the Bean attributes/operations so they can actually specialize or override a specific business functionality..." [Interceptors and Decorators tutorial]

CDI is one of the most important and popular parts of the Java EE. That’s not surprising.
I hope you enjoyed the example.

You can find the example on GithubGitHub-Mark-32px

 

Resources

CDI http://www.cdi-spec.org/

Weld http://weld.cdi-spec.org/

Reference Implementation Weld http://docs.jboss.org/weld/reference/latest/en-US/html/

Interceptor and Decorator Tutorial http://www.mastertheboss.com/jboss-frameworks/cdi/interceptors-and-decorators-tutorial?start=1

Contexts and Dependency Injection for Java EE  https://docs.oracle.com/javaee/7/tutorial/partcdi.htm#GJBNR

Test JSON with a few lines of code

JSONassert helps you writing JSON unit tests

You have to write a unit test and JSON is involved. Usually this will end up in a lot of lines of code. You know the situation, I bet.

Here JSONassert comes into play. It allows to write JSON unit tests with just a few lines of code. For example checking an JSON object:

[code]
book = new Book();
book.setName("The book");

String result = BookUtils.toJson(book);
boolean strict = false;
JSONAssert.assertEquals("{\"name\":\"The book\"}", result, strict);
[/code]

The strictMode == false makes ordering not relevant and makes your test less brittle.

JSONassert comes with JSONCompareResult. I thinks this class is very handy when you are interested in a particular aspect like a missing field or value.

[code]

JSONCompareResult result =
JSONCompare.compareJSON("{\"name\":\"The book\",\"pages\":100}",
"{\"name\":\"The book\"}", JSONCompareMode.LENIENT);
assertTrue(result.isMissingOnField());
assertEquals(result.getFieldMissing().size(), 1);
FieldComparisonFailure fieldComparisonFailure = result.getFieldMissing().get(0);
assertEquals("pages",fieldComparisonFailure.getExpected());
[/code]

JSONCompareResult provides a list of missing fields and fields with errors. Depending on your case, you can easily determine which field makes trouble.

JSONassert is a real cool third library. I like it!

GitHub-Mark-32px On Github you find more examples.

 

Resources

http://jsonassert.skyscreamer.org/quickstart.html
http://jsonassert.skyscreamer.org/apidocs/index.html
https://github.com/skyscreamer/JSONassert

Are we doing microservices?

Recently, an admin asked me if we are doing microservices.

I thought about it for a few moments. And what I said comes here as a checklist.
It’s a really simple test with quick yes or no.

Single Responsibility Principle
Is your service small and focused on doing one thing well?

Domain-Driven Design
Is your service modeled around business concepts?

Technology Heterogeneity
Are you free to pick technology, tool, language, framework or what ever for your service?

Interface
Has your service an explicitly published interface and
are internal implementation details are hidden?

Autonomous
Can you make a change to a service without changing anything else?

Independently deployments
Requires a deployment of your service coordination with other teams?

Scaling
Can you scale your service independently via cloning or sharding?

Resilence
Does your service can take corrective actions automatically?

Fault and resource isolation
Will affect a memory leak or an unclosed database connection only your service?

Lightweight Communication
Do you use REST or other lightweight communication protocols?

Getting Eclipse Neon running under Kubuntu (14.04 LTS)

Here I write about the pitfalls getting Eclipse Neon running

Black tooltips

First issue was an easy one. I fixed the behaviour as follows:

System Settings > Common Appearance and Behaviour > 
Application Appearance > Colours > Colour >
Tooltip Background > Tooltip Text

Choose a different color and click apply.

 

Eclipse crashes opening some dialogs and missing main menu 

Fixing this issues took me longer than I expected. Every time I launched Eclipse Marketplace the application crashed. And the missing main menu was very annoying.

I found out than the reason lies in a KDE bug.

Edit your eclipse.ini adding

[code]
–launcher.GTK_version 2
[/code]

or set these variable, e.g. in your application launcher file

[code]
export SWT_GTK3=0
[/code]

 

Further I edited the file „/usr/share/themes/oxygen-gtk/gtk-2.0/gtkrc“ and changed

[code]
GtkComboBox::appears-as-list = 1
[/code]

into

[code]
GtkComboBox::appears-as-list = 0
[/code]

 

Application Launcher file and Eclipse configugration

So my application launcher file and Eclipse configuration file looks like this.

~/local/share/applications/Eclipse-Neon.desktop

[code]
[Desktop Entry]
Name=Eclipse
Type=Application
Exec=SWT_GTK3=0 /home/xyz/eclipse/jee-neon/eclipse/eclipse -clean -vm /usr/lib/jvm/oracle-java8-jdk-amd64/bin/java
Terminal=false
Icon=/home/xyz/eclipse/jee-neon/eclipse/icon.xpm
Comment=Integrated Development Environment
NoDisplay=false
Categories=Development;IDE;Java;
Name[en]=Eclipse
[/code]

eclipse.ini

[code]
-startup
plugins/org.eclipse.equinox.launcher_1.3.200.v20160318-1642.jar
–launcher.library
plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.400.v20160518-1444
-product
org.eclipse.epp.package.jee.product
-showsplash
org.eclipse.platform
–launcher.defaultAction
openFile
–launcher.appendVmargs
-vmargs
-Dosgi.requiredJavaVersion=1.8
-XX:+UseG1GC
-XX:+UseStringDeduplication
-Xms256m
-Xmx1024m
[/code]

Now, I am happy 😉

Links

http://askubuntu.com/a/258451
https://bugs.eclipse.org/bugs/show_bug.cgi?id=440660#c20
https://bugs.kde.org/show_bug.cgi?id=339174#c16
https://bugs.eclipse.org/bugs/show_bug.cgi?id=430736#c50

Reading YAML configurations in Java

How can I use YAML for my Java application?

YAML is human-readable and a cool datat interchange format. It’s lean appearance make it special e.g. for configuration files or when humans view or edit data structures. It’s well suited for hierachical data presentation and you can easily map common data structures.

Jackson and SnakeYaml

Here I present two YAML libaries: Jackson (Dataformat YAML extension) and Snakeyaml. JSON is a basis of YAML, so you can use Jackson for your YAML cases.
Snakeyaml is a only YAML library. I start the Jackson example.

My configuration file looks like this.

[code]
server:
type: simple
id: abc
timeoutMs: 100
database:
user: sam
password: sum
url: ‚jdbc:mysql://localhost:3306/db‘
[/code]

For every part in my configuration I have written separate beans e.g. Server, Database. And my class Configuration includes all these beans.

For Reading a YAML configuration file with Jackson you need to do the following:

[code]
ObjectMapper objectMapper = new ObjectMapper(new YAMLFactory());
InputStream  configurationFile = …
Configuration configuration = objectMapper.readValue(configurationFile,Configuration.class);
[/code]

That’s all. Pretty easy, isn’t it?

Now, let’s turn to the Snakeyaml library. With Snakeyaml you also need just a few lines.

[code]
Yaml yaml = new Yaml();
InputStream configurationFile = …;
Configuration configuration = (Configuration) yaml.load(configurationFile);
[/code]

Lesson learned

Both libaries are really easy to use. Snakeyaml has a large feature set, a good documentation and the fingerprint is small (<250 kb). So if you only interested in reading a YAML configuration and you are not have anything to do with JSON, go with Snakeyaml.

GitHub-Mark-32px On Github you also find examples for writing out YAML files.

 

Resources

http://yaml.org/
https://github.com/FasterXML/jackson-dataformat-yaml
https://bitbucket.org/asomov/snakeyaml

Initializing HashMaps in Java

How can I initialize a HashMap with some pre-defined entries?

In this article I’ll show you how you create a HashMap with values. Java doesn’t provide a method like Array.asList(). For initializing a map you have to implement something like that:

[code]
//Old fashion way
Map<Integer, String> oldFashion = new HashMap<Integer, String>() {{
put(1, "one");
put(2, "two");
}};
[/code]

So this is the double brace initialization idom. I don’t like this way. It’s not very readable, it’s creates a new class every time and may cause a memory leak. That’s may be okay for tests.

But as you know there isn’t just one way of doing it. With Java 8 you can solve my problem with a Lambda Expression.

[code]
//Java 8 Lambda way
Map<Integer, String> lambdaMap = Stream.of(
new SimpleEntry<Integer, String>(1, "one"),
new SimpleEntry<Integer, String>(10, "ten"))
.collect(
Collectors.toMap((se) -> se.getKey(), (se) -> se.getValue())
);
[/code]

There are new methods added to java.util.Arrays to convert an array into a java.util.stream.Stream which can be used for summing, multiplying or collecting. Mmmh, I think that’s a lot of code for just initializing a Map with values.

Okay, let’s see what Guava can do for us.

[code]
//Guava way
ImmutableMap<Integer, String> guavaMap = ImmutableMap.of(
1, "one",
2, "two"
);

//Guava way with builder
ImmutableMap<Integer, String> guavaMap2 = ImmutableMap.<Integer, String> builder()
.put(1, "one")
.put(2, "two")
.put(15, "fifteen")
.build();
[/code]

Guava has a Factory Method ImmutableMap.of() which returns a immutable map containing our values. The Library has also a ImmutableMap.Builder<K,V>.build() method.

Lesson Learned

The Guava way is my favorite. It’s typed, safe, short and readable. If you are afraid of Guava, then consider the plain old way. The Java 8 Lambda Stream way is pretty ugly.

GitHub-Mark-32px Examples on Github

Stability Patterns: Use Timeouts

„A resilient system keeps processing transactions, even when there are transient impulses, persistent stresses, or component failure disrupting normal processing.“

This is Michael Nygards definition of stability. In his book „Release it!“ he describes design and architectures patterns, which stop cracks from propagating and preserve at least partial functionality instead of total crashes.

So, what are the problems?

Why do distributed systems crash? The fallacies are (from Fallacies of distributed computing):

The network is reliable.
Latency is zero.
Bandwidth is infinite.
The network is secure.
Topology doesn’t change.
There is one administrator.
Transport cost is zero.
The network is homogeneous.

Networks are not reliable and latency is not zero. That’s why at every point a subsystem is integrated, a call could eventually fail. Without timeouts upstream systems will themselves slow down and might be vulnerable to stability problems. And failures in remote systems propagate quickly, probably turning into an cascading failure.

Failure Modes

Accept that failures could happen and design how your system reacts to specific failures. This is what Nygard calls failure modes. A failure mode contain damage and protect the rest of a system, a sort of self-protection which determines the resilience of a whole system. These modes are like crumple zones, areas designed to protect passengers by failing fast. So think about your system. Which features are indispensable? And then build failure modes around them.

Using Timeouts

Using Timeouts is one pattern you can use to defend such failures in your system. If you look at different Java Specifications, e.g JDBC, JMS or JAX-RS you can find methods which have a timeout parameter. Or with Jersey you can easily set a timeout at the client side like this:

[code]
Jersey HttpUrlConnection
restClient = ClientBuilder.newClient();
restClient.property(ClientProperties.CONNECT_TIMEOUT, 2000); 
restClient.property(ClientProperties.READ_TIMEOUT, 2000);
[/code]

Why are here two timeout properties? The Sockets API defines two types of timeouts: connection timeout defines a maximum time elapsed before the connection is established or an error occurs, and socket timeout determines the maximum period of inactivity between two consecutive data packets arriving on the client side after a connection has been established.

Okay, and how can I timeout a call when a third library doesn’t provide a method with a timeout parameter?

[code]
ExecutorService executorService = Executors.newSingleThreadExecutor();

Future<String> task = executorService.submit(new Callable<String>() {
@Override
public String call() throws Exception {
long heavyTaskInMs = 6000;
Thread.sleep(heavyTaskInMs);
return "Hello Timeout";
}
});

try {
long timeoutInSeconds = 5;
System.out.println(task.get(timeoutInSeconds, TimeUnit.SECONDS));
} catch (InterruptedException | ExecutionException | TimeoutException e) {
e.printStackTrace();
}
[/code]

But wait! Don’t reinvent the wheel. You could use Guava’s SimpleTimeLimiter to solve that problem 😉

Defend with Timeouts

Always use methods that takes a timeout parameter or if not provided, make sure that your call will come back.

 GitHub-Mark-32px Examples on Github

Consent Management Platform von Real Cookie Banner