I have found that in order to understand covariance and contravariance a few examples with Java arrays are always a good start. Arrays Are Covariant Arrays are said to be covariant which basically means that, given the subtyping rules of Java, an array of type T[] may contain elements of type T or any subtype of T. For instance: Number[] numbers = newNumber[3]; numbers[0] = newInteger(10); numbers[1] = newDouble(3.14); numbers[2] = newByte(0); But not only that, the subtyping rules of Java also state that an array S[] is a subtype of the array T[] if S is a subtype of T, therefore, something like this is also valid: Integer[] myInts = {1,2,3,4}; Number[] myNumber = myInts; Because according to the subtyping rules in Java, an array Integer[] is a subtype of an array Number[] because Integer is a subtype of Number. But this subtyping rule can lead to an interesting question: what would happen if we try to do this? myNumber[0] = 3.14; //attempt of heap pollution This last line would compile just fine, but if we run this code, we would get an ArrayStoreException because we’re trying to put a double into an integer array. The fact that we are accessing the array through a Number reference is irrelevant here, what matters is that the array is an array of integers. This means that we can fool the compiler, but we cannot fool the run-time type system. And this is so because arrays are what we call a reifiable type. This means that at run-time Java knows that this array was actually instantiated as an array of integers which simply happens to be accessed through a reference of type Number[]. So, as we can see, one thing is the actual type of the object, an another thing is the type of the reference that we use to access it, right? The Problem with Java Generics Now, the problem with generic types in Java is that the type information for type parameters is discarded by the compiler after the compilation of code is done; therefore this type information is not available at run time. This process is called type erasure. There are good reasons for implementing generics like this in Java, but that’s a long story, and it has to do with binary compatibility with pre-existing code. The important point here is that since at run-time there is no type information, there is no way to ensure that we are not committing heap pollution. Let’s consider now the following unsafe code: List myInts = newArrayList(); myInts.add(1); myInts.add(2); List myNums = myInts; //compiler error myNums.add(3.14); //heap polution If the Java compiler does not stop us from doing this, the run-time type system cannot stop us either, because there is no way, at run time, to determine that this list was supposed to be a list of integers only. The Java run-time would let us put whatever we want into this list, when it should only contain integers, because when it was created, it was declared as a list of integers. That’s why the compiler rejects line number 4 because it is unsafe and if allowed could break the assumptions of the type system. As such, the designers of Java made sure that we cannot fool the compiler. If we cannot fool the compiler (as we can do with arrays) then we cannot fool the run-time type system either. As such, we say that generic types are non-reifiable, since at run time we cannot determine the true nature of the generic type. Evidently this property of generic types in Java would have a negative impact on polymorphism. Let’s consider now the following example: staticlongsum(Number[] numbers) { longsummation = 0; for(Number number : numbers) { summation += number.longValue(); } returnsummation; } Now we could use this code as follows: Integer[] myInts = {1,2,3,4,5}; Long[] myLongs = {1L, 2L, 3L, 4L, 5L}; Double[] myDoubles = {1.0, 2.0, 3.0, 4.0, 5.0}; System.out.println(sum(myInts)); System.out.println(sum(myLongs)); System.out.println(sum(myDoubles)); But if we attempt to implement the same code with generic collections, we would not succeed: staticlongsum(List numbers) { longsummation = 0; for(Number number : numbers) { summation += number.longValue(); } returnsummation; } Because we we would get compiler errors if you try to do the following: List myInts = asList(1,2,3,4,5); List myLongs = asList(1L, 2L, 3L, 4L, 5L); List myDoubles = asList(1.0, 2.0, 3.0, 4.0, 5.0); System.out.println(sum(myInts)); //compiler error System.out.println(sum(myLongs)); //compiler error System.out.println(sum(myDoubles)); //compiler error The problem is that now we cannot consider a list of integers to be subtype of a list of numbers, as we saw above, that would be considered unsafe for the type system and compiler rejects it immediately. Evidently, this is affecting the power of polymorphism and it needs to be fixed. The solution consists in learning how to use two powerful features of Java generics known as covariance and contravariance. Covariance For this case, instead of using a type T as the type argument of a given generic type, we use a wildcard declared as ? extends T, where T is a known base type. With covariance we can read items from a structure, but we cannot write anything into it. All these are valid covariant declarations. List myNums = newArrayList(); List myNums = newArrayList(); List myNums = newArrayList(); And we can read from our generic structure myNums by doing: Number n = myNums.get(0); Because we can be sure that whatever the actual list contains, it can be upcasted to a Number (after all anything that extends Number is a Number, right?) However, we are not allowed to put anything into a covariant structure. myNumst.add(45L); //compiler error This would not be allowed because the compiler cannot determine what is the actual type of the object in the generic structure. It can be anything that extends Number (like Integer, Double, Long), but the compiler cannot be sure what, and therefore any attempt to retrieve a generic value is considered an unsafe operation and it is immediately rejected by the compiler. So we can read, but not write. Contravariance For contravariance we use a different wildcard called ? super T, where T is our base type. With contravariance we can do the opposite. We can put things into a generic structure, but we cannot read anything out of it. In this case, the actual nature of the object is List of Object, and through contravariance, we can put a Number in it, basically because a Number has Object as its common ancestor. As such, all numbers are also objects, and therefore this is valid. However, we cannot safely read anything from this contravariant structure assuming that we will get a number. Number myNum = myNums.get(0); //compiler-error As we can see, if the compiler allowed us to write this line, we would get a ClassCastException at run time. So, once again, the compiler does not run the risk of allowing this unsafe operation and rejects it immediately. Get/Put Principle In summary, we use covariance when we only intend to take generic values out of a structure. We use contravariance when we only intend to put generic values into a structure and we use an invariant when we intend to do both. The best example I have is the following that copies any kind of numbers from one list into another list. It only gets items from the source, and it only puts items in the destiny. publicstaticvoidcopy(List source, List destiny) { for(Number number : source) { destiny.add(number); } } Thanks to the powers of covariance and contravariance this works for a case like this: List myInts = asList(1,2,3,4); List myDoubles = asList(3.14, 6.28); List

I had previously written about functional interfaces and their usage. If you are exploring the APIs to be part of Java 8 and especially those APIs which support lambda expressions you will find few interfaces like- Function, Supplier, Consumer, Predicate and others which are all part of the java.util.function package, being used extensively. These interfaces have one abstract method, which is overridden by the lambda expression defined. In this post I will pick Function interface to explain about it in brief and it is one of the interfaces present in java.util.function package. Function interface has two methods: R apply(T t) – Compute the result of applying the function to the input argument default ‹V› Function‹T,V› – Combine with another function returning a function which performs both functions. In this post I would like to write about the apply method, creating APIs which accept these interfaces and parameters and then invoke their corresponding methods. We will also look at how the caller of the API can pass in a lambda expression in place of an implementation of the interface. Apart from passing a lambda expression, the users of the API can also pass method references, about which I havent blogged yet. Function interface is uses in cases where you want to encapsulate some code into a method which accepts some value as an input parameter and then returns another value after performing required operations on the input. The input parameter type and the return type of the method can either be same or different. Lets look at an API which accepts an implementation of Function interface: public class FunctionDemo { //API which accepts an implementation of //Function interface static void modifyTheValue(int valueToBeOperated, Function function){ int newValue = function.apply(valueToBeOperated); /* * Do some operations using the new value. */ System.out.println(newValue); } } Now lets look at the code which invokes this API: public static void main(String[] args) { int incr = 20; int myNumber = 10; modifyTheValue(myNumber, val-> val + incr); myNumber = 15; modifyTheValue(myNumber, val-> val * 10); modifyTheValue(myNumber, val-> val - 100); modifyTheValue(myNumber, val-> "somestring".length() + val - 100); } You can see that the lambda expressions being created accept one parameter and return some value. I will update soon about the various APIs which use this Function interface as a parameter. Meanwhile the complete code is: public class FunctionDemo { public static void main(String[] args) { int incr = 20; int myNumber = 10; modifyTheValue(myNumber, val-> val + incr); myNumber = 15; modifyTheValue(myNumber, val-> val * 10); modifyTheValue(myNumber, val-> val - 100); modifyTheValue(myNumber, val-> "somestring".length() + val - 100); } //API which accepts an implementation of //Function interface static void modifyTheValue(int valueToBeOperated, Function function){ int newValue = function.apply(valueToBeOperated); /* * Do some operations using the new value. */ System.out.println(newValue); } } and the output is: 30 150 -85 -75 In the coming posts I will try to explore the other interfaces present in java.util.function package. Note: The above code was compiled using the JDK downloaded from here and Netbeans 8 nightly builds.

Some of you might be aware that the Public Final Draft version of Java EE 7 spec has been released. Among various other new things, this version of Java EE, brings in EJB 3.2 version of the EJB specification. EJB 3.2 has some new features compared to the EJB 3.1 spec. I'm quoting here the text present in the EJB 3.2 spec summarizing what's new: The Enterprise JavaBeans 3.2 architecture extends Enterprise JavaBeans to include the following new functionality and simplifications to the earlier EJB APIs: Support for the following features has been made optional in this release and their description is moved to a separate EJB Optional Features document: EJB 2.1 and earlier Entity Bean Component Contract for Container-Managed Persistence EJB 2.1 and earlier Entity Bean Component Contract for Bean-Managed Persistence Client View of an EJB 2.1 and earlier Entity Bean EJB QL: Query Language for Container-Managed Persistence Query Methods JAX-RPC Based Web Service Endpoints JAX-RPC Web Service Client View Added support for local asynchronous session bean invocations and non-persistent EJB Timer Service to EJB 3.2 Lite. Removed restriction on obtaining the current class loader; replaced ‘must not’ with ‘should exercise caution’ when using the Java I/O package. Added an option for the lifecycle callback interceptor methods of stateful session beans to be executed in a transaction context determined by the lifecycle callback method's transaction attribute. Added an option to disable passivation of stateful session beans. Extended the TimerService API to query all active timers in the same EJB module. Removed restrictions on javax.ejb.Timer and javax.ejb.TimerHandle references to be used only inside a bean. Relaxed default rules for designating implemented interfaces for a session bean as local or as remote business interfaces. Enhanced the list of standard activation properties. Enhanced embeddable EJBContainer by implementing AutoClosable interface. As can be seen, some of the changes proposed are minor. But there are some which are useful major changes. We'll have a look at a couple of such changes in this article. 1) New API TimerService.getAllTimers() EJB 3.2 version introduces a new method on the javax.ejb.TimerService interface, named getAllTimers. Previously the TimerService interface had (and still has) a getTimers method. The getTimers method was expected to return the active timers that are applicable for the bean on whose TimerService, the method had been invoked (remember: there's one TimerService per EJB). In this new EJB 3.2 version, the newly added getAllTimers() method is expected to return all the active timers that are applicable to *all beans within the same EJB module*. Typically, an EJB module corresponds to a EJB jar, but it could also be a .war deployment if the EJBs are packaged within the .war. This new getAllTimers() method is a convenience API for user applications which need to find all the active timers within the EJB module to which that bean belongs. 2) Ability to disable passivation of stateful beans Those familiar with EJBs will know that the EJB container provides passivation (storing the state of the stateful bean to some secondary store) and activation (loading the saved state of the stateful bean) capability to stateful beans. However, previous EJB versions didn't have a portable way of disabling passivation of stateful beans, if the user application desired to do that. The new EJB 3.2 version introduces a way where the user application can decide whether the stateful bean can be passivated or not. By default, the stateful bean is considered to be "passivation capable" (like older versions of EJB). However, if the user wants to disable passivation support for certain stateful bean, then the user has the option to either disable it via annotation or via the ejb-jar.xml deployment descriptor. Doing it the annotation way is as simple as setting the passivationCapable attribute on the @javax.ejb.Stateful annotation to false. Something like: @javax.ejb.Stateful(passivationCapable=false) // the passivationCapable attribute takes a boolean value public class MyStatefulBean { .... } Doing it in the ejb-jar.xml is as follows: foo-bar-bean org.myapp.FooBarStatefulBean Stateful false ... Two important things to note in that ejb-jar.xml are the version=3.2 attribute (along with the http://xmlns.jcp.org/xml/ns/javaee/ejb-jar_3_2.xsd schema location) on the ejb-jar root element and the passivation-capable element under the session element. So, using either of these approaches will allow you to disable passivation on stateful beans, if you want to do so. Java EE 7 and EJB 3.2 support in JBoss AS8: JBoss AS8 has been adding support for Java EE 7 since the Public Final Draft version of the spec has been announced. Support for EJB 3.2 is already added and made available. Some other Java EE 7 changes have also made it to the latest JBoss AS 8 builds. To keep track of the Java EE 7 changes in JBoss AS8, keep an eye on this JIRA https://issues.jboss.org/browse/AS7-6553. To use the already implemented features of Java EE 7 in general or EJB 3.2 in particular, you can download the latest nightly build/binary of JBoss AS from here. Give it a try and let us know how it goes. For any feedback, questions or if you run into any kind of issues, feel free to open a discussion thread in our user forum here.

Generally, you pick up a subset of components in some integration tests to check if they are glued as expected. To achieve this, they are usually really invoked, but sometimes, it is too expensive to do so. For example, Component A invokes Component B, and Component B has a dependency on an external system which does not have a test server. We really want to verify the configurations, it seems the only way is replacing Component B with test double after wiring Component A and B. Let's start with Strategy A: Manual Injecting @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = "classpath:config.xml") public class SomeAppIntegrationTestsUsingManualReplacing { private Mockery context = new JUnit4Mockery(); （1） private SomeInterface mock = context.mock(SomeInterface.class); （2） @Resource(name = "someApp") private SomeApp someApp; （3） @Before public void replaceDependenceWithMock() { someApp.setDependence(mock); （4） } @DirtiesContext @Test public void returnsHelloWorldIfDependenceIsAvailable() throws Exception { context.checking(new Expectations() { { allowing(mock).isAvailable(); will(returnValue(true)); （5） } }); String actual = someApp.returnHelloWorld(); assertEquals("helloWorld", actual); context.assertIsSatisfied(); （6） } } We get a spring bean someApp(Component A in this case), and it has a denpendence on SomeInterface's(Component B in this case). We inject mock (declare and init at step 4) to someApp, thus the test passes without sending request to the external system. The context.assertIsSatisfied()(at step 6 ) is very important as we use SpringJUnit4ClassRunner as junit runner instead of JMock, so you have to explictly assert that all expectations are satisfied. There are two downsides of the previous strategy: Firstly, if there are more than one mock, you have to inject them one by one, which is very tedious especially when you need to inject mocks into serveral spring bean. Secondly, the wiring is not tested. For example, if I forget to write the integration tests using manual inject strategy is not going to tell. Strategy B: Using predefined BeanPostProcessor Spring provides BeanPostProcessor which is very useful when you want to replace some bean after the wiring is done. According to the reference, application context will auto detect all BeanPostProcessor registered in metadata(usually in xml format). public class PredefinedBeanPostProcessor implements BeanPostProcessor { public Mockery context = new JUnit4Mockery(); （1） public SomeInterface mock = context.mock(SomeInterface.class); （2） @Override public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException { return bean; } @Override public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException { if ("dependence".equals(beanName)) { return mock; } else { return bean; } } } @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = { "classpath:config.xml", "classpath:predefined.xml" }) （1） public class SomeAppIntegrationTestsUsingPredefinedReplacing { @Resource(name = "someApp") private SomeApp someApp; @Resource(name = "predefined") private PredefinedBeanPostProcessor fixture; @Test public void returnsHelloWorldIfDependenceIsAvailable() throws Exception { fixture.context.checking(new Expectations() { { allowing(fixture.mock).isAvailable(); will(returnValue(true)); } }); String actual = someApp.returnHelloWorld(); assertEquals("helloWorld", actual); fixture.context.assertIsSatisfied(); } } Notice there is an extra config xml in which the PredefinedBeanPostProcessor is registered(at step 1). The predefined.xml is placed in src/test/resources/, so it will not be packed into the artifact for production. For each test, using Strategy B requires inputting both a java file and a xml which is quite verbose. Now we have learned the pros and cons of Strategy A and Strategy B. What about a hybrid version -- killing two birds with one stone. Therefore we have the next strategy. Strategy C：Dynamic Injecting public class TestDoubleInjector implements BeanPostProcessor { private static Map MOCKS = new HashMap(); （1） @Override public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException { return bean; } @Override public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException { if (MOCKS.containsKey(beanName)) { return MOCKS.get(beanName); } return bean; } public void addMock(String beanName, Object mock) { MOCKS.put(beanName, mock); } public void clear() { MOCKS.clear(); } } @RunWith(JMock.class) public class SomeAppIntegrationTestsUsingDynamicReplacing { private Mockery context = new JUnit4Mockery(); private SomeInterface mock = context.mock(SomeInterface.class); private SomeApp someApp; private ConfigurableApplicationContext applicationContext; private TestDoubleInjector fixture = new TestDoubleInjector(); （1） @Before public void replaceDependenceWithMock() { fixture.addMock("dependence", mock); （2） applicationContext = new ClassPathXmlApplicationContext(new String[] { "classpath:config.xml", "classpath:dynamic.xml" }); （3） someApp = (SomeApp) applicationContext.getBean("someApp"); } @Test public void returnsHelloWorldIfDependenceIsAvailable() throws Exception { context.checking(new Expectations() { { allowing(mock).isAvailable(); will(returnValue(true)); } }); String actual = someApp.returnHelloWorld(); assertEquals("helloWorld", actual); } @After public void clean() { applicationContext.close(); fixture.clear(); } } The TestDoubleInjector class is an implementation of Monostate pattern. Mocks are added to the static map before the application context being created. When another TestDoubleInjector instance (defined in dynamic.xml) is initiated, it can share the static map for replacement. Just beware to clear the static map after tests. By the way, you could use Stub instead of Mocks with same strategies. Please do not hesitate to contact me if you might have any questions. And I do appreciate it, if you could let me know you have a better idea. Thanks! Resources： http://www.jmock.org http://www.oracle.com/technetwork/articles/entarch/spring-aop-with-ejb5-093994.html(I saw BeanPostProcessor the first time in this post)

Clojure, being designed for concurrency is a natural fit for our Back to the Future series. Moreover futures are supported out-of-the-box in Clojure. Last but not least, Clojure is the first language/library that draws a clear distinction between futures and promises. They are so similar that most platforms either support only futures or combine them. Clojure is very explicit here, which is good. Let's start from promises: Promises Promise is a thread-safe object that encapsulates immutable value. This value might not be available yet and can be delivered exactly once, from any thread, later. If other thread tries to dereference a promise before it's delivered, it'll block calling thread. If promise is already resolved (delivered), no blocking occurs. Promise can only be delivered once and can never change its value once set: (def answer (promise)) @answer (deliver answer 42) answer is a promise var. Trying to dereference it using @answer or (deref answer) at this point will simply block. This or some other thread must first deliver some value to this promise (using deliver function). All threads blocked on deref will wake up and subsequent attempts to dereference this promise will return 42 immediately. Promise is thread safe and you cannot modify it later. Trying to deliver another value to answer is ignored. Futures Futures behave pretty much the same way in Clojure from user perspective - they are containers for a single value (of course it can be a map or list - but it should be immutable) and trying to dereference future before it is resolved blocks. Also just like promises, futures can only be resolved once and dereferencing resolved future has immediate effect. The difference between the two is semantic, not technical. Future represents background computation, typically in a thread pool while promise is just a simple container that can be delivered (filled) by anyone at any point in time. Typically there is no associated background processing or computation. It's more like an event we are waiting for (e.g. JMS message reply we wait for). That being said, let's start some asynchronous processing. Similar to Akka, underlying thread pool is implicit and we simply pass piece of code that we want to run in background. For example to calculate the sum of positive integers below ten million we can say: (let [sum (future (apply + (range 1e7)))] (println "Started...") (println "Done: " @sum) ) sum is the future instance. "Started..." message appears immediately as the computation started in background thread. But @sum is blocking and we actually have to wait a little bit1 to see the "Done: " message and computation results. And here is where the greatest disappointment arrives: neither future nor promise in Clojure supports listening for completion/failure asynchronously. The API is pretty much equivalent to very limited java.util.concurrent.Future. We can create future, cancel it, check whether it is realized? (resolved) and block waiting for a value. Just like Future in Java, as a matter of fact the result of future function even implements java.util.concurrent.Future. As much as I love Clojure concurrency primitives like STM and agents, futures feel a bit underdeveloped. Lack of event-driven, asynchronous callbacks that are invoked whenever futures completes (notice that add-watch doesn't work futures - and is still in alpha) greatly reduces the usefulness of a future object. We can no longer: map futures to transform result value asynchronously chain futures translate list of futures to future of list ...and much more, see how Akka does it and Guava to some extent That's a shame and since it's not a technical difficulty but only a missing API, I hope to see support for completion listeners soon. For completeness here is a slightly bigger program using futures to concurrently fetch contents of several websites, foundation for our web crawling sample: (let [ top-sites `("www.google.com" "www.youtube.com" "www.yahoo.com" "www.msn.com") futures-list (doall ( map #( future (slurp (str "http://" %)) ) top-sites )) contents (map deref futures-list) ] (doseq [s contents] (println s)) ) Code above starts downloading contents of several websites concurrently. map deref waits for all results one after another and once all futures from futures-list all completed, doseq prints the contents (contents is a list of strings). One trap I felt into was the absence of doall (that forces lazy sequence evaluation) in my initial attempt. map produces lazy sequence out of top-sites list, which means future function is called only when given item of futures-list is first accessed. That's good. But each item is accessed for the first time only during (map deref futures-list). This means that while waiting for first future to dereference, second future didn't even started yet! It starts when first future completes and we try to dereference the second one. That means that last future starts when all previous futures are already completed. To cut long story short, without doall that forces all futures to start immediately, our code runs sequentially, one future after another. The beauty of side effects. 1 - BTW (1L to 9999999L).sum in Scala is faster by almost an order of magnitude, just sayin'...

Aliasing means there are multiple aliases to a location that can be updated, and these aliases have different types. In the following example, a and b are two variable names that have two different types A and B. B extends A. B[] b = new B[10]; A[] a = b; a[0] = new A(); b[0].methodParent(); In memory, they both refer to the same location. The pointed memory location are pointed by both a and b. During run-time, the actual object stored determines which method to call. How does Java handle aliasing problem? If you copy this code to your eclipse, there will be no compilation errors. class A { public void methodParent() { System.out.println("method in Parent"); } } class B extends A { public void methodParent() { System.out.println("override method in Child"); } public void methodChild() { System.out.println("method in Child"); } } public class Main { public static void main(String[] args) { B[] b = new B[10]; A[] a = b; a[0] = new A(); b[0].methodParent(); } } But if you run the code, the output would be as follows: Exception in thread “main” java.lang.ArrayStoreException: aliasingtest.A at aliasingtest.Main.main(Main.java:26) The reason is that Java handles aliasing during run-time. During run-time, it knows that the first element should be a B object, instead of A. Therefore, it only runs correctly if it is changed to: B[] b = new B[10]; A[] a = b; a[0] = new B(); b[0].methodParent(); and the output is: override method in Child * original article

in a set, there are no duplicate elements. that is one of the major reasons to use a set. there are 3 commonly used implementations of set in java: hashset, treeset and linkedhashset. when and which to use is an important question. in brief, if we want a fast set, we should use hashset; if we need a sorted set, then treeset should be used; if we want a set that can be read by following its insertion order, linkedhashset should be used. 1. set interface set interface extends collection interface. in a set, no duplicates are allowed. every element in a set must be unique. we can simply add elements to a set, and finally we will get a set of elements with duplicates removed automatically. 2. hashset vs. treeset vs. linkedhashset hashset is implemented using a hash table. elements are not ordered. the add, remove, and contains methods has constant time complexity o(1). treeset is implemented using a tree structure(red-black tree in algorithm book). the elements in a set are sorted, but the add, remove, and contains methods has time complexity of o(log (n)). it offers several methods to deal with the ordered set like first(), last(), headset(), tailset(), etc. linkedhashset is between hashset and treeset. it is implemented as a hash table with a linked list running through it, so it provides the order of insertion. the time complexity of basic methods is o(1). 3. treeset example treeset tree = new treeset(); tree.add(12); tree.add(63); tree.add(34); tree.add(45); iterator iterator = tree.iterator(); system.out.print("tree set data: "); while (iterator.hasnext()) { system.out.print(iterator.next() + " "); } output is sorted as follows: tree set data: 12 34 45 63 now let's define a dog class as follows: class dog { int size; public dog(int s) { size = s; } public string tostring() { return size + ""; } } let's add some dogs to treeset like the following: import java.util.iterator; import java.util.treeset; public class testtreeset { public static void main(string[] args) { treeset dset = new treeset(); dset.add(new dog(2)); dset.add(new dog(1)); dset.add(new dog(3)); iterator iterator = dset.iterator(); while (iterator.hasnext()) { system.out.print(iterator.next() + " "); } } } compile ok, but run-time error occurs: exception in thread "main" java.lang.classcastexception: collection.dog cannot be cast to java.lang.comparable at java.util.treemap.put(unknown source) at java.util.treeset.add(unknown source) at collection.testtreeset.main(testtreeset.java:22) because treeset is sorted, the dog object need to implement java.lang.comparable's compareto() method like the following: class dog implements comparable{ int size; public dog(int s) { size = s; } public string tostring() { return size + ""; } @override public int compareto(dog o) { return size - o.size; } } the output is: 1 2 3 4. hashset example hashset dset = new hashset(); dset.add(new dog(2)); dset.add(new dog(1)); dset.add(new dog(3)); dset.add(new dog(5)); dset.add(new dog(4)); iterator iterator = dset.iterator(); while (iterator.hasnext()) { system.out.print(iterator.next() + " "); } output: 5 3 2 1 4 note the order is not certain. 5. linkedhashset example linkedhashset dset = new linkedhashset(); dset.add(new dog(2)); dset.add(new dog(1)); dset.add(new dog(3)); dset.add(new dog(5)); dset.add(new dog(4)); iterator iterator = dset.iterator(); while (iterator.hasnext()) { system.out.print(iterator.next() + " "); } the order of the output is certain and it is the insertion order. 2 1 3 5 4 6. performance testing the following method tests the performance of the three class on add() method. public static void main(string[] args) { random r = new random(); hashset hashset = new hashset(); treeset treeset = new treeset(); linkedhashset linkedset = new linkedhashset(); // start time long starttime = system.nanotime(); for (int i = 0; i < 1000; i++) { int x = r.nextint(1000 - 10) + 10; hashset.add(new dog(x)); } // end time long endtime = system.nanotime(); long duration = endtime - starttime; system.out.println("hashset: " + duration); // start time starttime = system.nanotime(); for (int i = 0; i < 1000; i++) { int x = r.nextint(1000 - 10) + 10; treeset.add(new dog(x)); } // end time endtime = system.nanotime(); duration = endtime - starttime; system.out.println("treeset: " + duration); // start time starttime = system.nanotime(); for (int i = 0; i < 1000; i++) { int x = r.nextint(1000 - 10) + 10; linkedset.add(new dog(x)); } // end time endtime = system.nanotime(); duration = endtime - starttime; system.out.println("linkedhashset: " + duration); } from the output below, we can clearly wee that hashset is the fastest one. hashset: 2244768 treeset: 3549314 linkedhashset: 2263320 if you enjoyed this article and want to learn more about java collections, check out this collection of tutorials and articles on all things java collections.

Any Java developer around the world would have used at least one of the following interfaces: java.lang.Runnable, java.awt.event.ActionListener, java.util.Comparator, java.util.concurrent.Callable. There is some common feature among the stated interfaces and that feature is they have only one method declared in their interface definition. There are lot more such interfaces in JDK and also lot more created by java developers. These interfaces are also called Single Abstract Method interfaces (SAM Interfaces). And a popular way in which these are used is by creating Anonymous Inner classes using these interfaces, something like: public class AnonymousInnerClassTest { public static void main(String[] args) { new Thread(new Runnable() { @Override public void run() { System.out.println("A thread created and running ..."); } }).start(); } } With Java 8 the same concept of SAM interfaces is recreated and are called Functional interfaces. These can be represented using Lambda expressions, Method reference and constructor references(I will cover these two topics in the upcoming blog posts). There’s an annotation introduced- @FunctionalInterface which can be used for compiler level errors when the interface you have annotated is not a valid Functional Interface. Lets try to have a look at a simple functional interface with only one abstract method: @FunctionalInterface public interface SimpleFuncInterface { public void doWork(); } The interface can also declare the abstract methods from the java.lang.Object class, but still the interface can be called as a Functional Interface: @FunctionalInterface public interface SimpleFuncInterface { public void doWork(); public String toString(); public boolean equals(Object o); } Once you add another abstract method to the interface then the compiler/IDE will flag it as an error as shown in the screenshot below: Interface can extend another interface and in case the Interface it is extending in functional and it doesn’t declare any new abstract methods then the new interface is also functional. But an interface can have one abstract method and any number of default methods and the interface would still be called an functional interface. To get an idea of default methods please read here. @FunctionalInterface public interface ComplexFunctionalInterface extends SimpleFuncInterface { default public void doSomeWork(){ System.out.println("Doing some work in interface impl..."); } default public void doSomeOtherWork(){ System.out.println("Doing some other work in interface impl..."); } } The above interface is still a valid functional interface. Now lets see how we can use the lambda expression as against anonymous inner class for implementing functional interfaces: /* * Implementing the interface by creating an * anonymous inner class versus using * lambda expression. */ public class SimpleFunInterfaceTest { public static void main(String[] args) { carryOutWork(new SimpleFuncInterface() { @Override public void doWork() { System.out.println("Do work in SimpleFun impl..."); } }); carryOutWork(() -> System.out.println("Do work in lambda exp impl...")); } public static void carryOutWork(SimpleFuncInterface sfi){ sfi.doWork(); } } And the output would be … Do work in SimpleFun impl... Do work in lambda exp impl... In case you are using an IDE which supports the Java Lambda expression syntax(Netbeans 8 Nightly builds) then it provides an hint when you use an anonymous inner class as used above This was a brief introduction to the concept of functional interfaces in java 8 and also how they can be implemented using Lambda expressions.

We all know that interfaces in Java contain only method declarations and no implementations and any non-abstract class implementing the interface had to provide the implementation. Lets look at an example: public interface SimpleInterface { public void doSomeWork(); } class SimpleInterfaceImpl implements SimpleInterface{ @Override public void doSomeWork() { System.out.println("Do Some Work implementation in the class"); } public static void main(String[] args) { SimpleInterfaceImpl simpObj = new SimpleInterfaceImpl(); simpObj.doSomeWork(); } } Now what if I add a new method in the SimpleInterface? public interface SimpleInterface { public void doSomeWork(); public void doSomeOtherWork(); } and if we try to compile the code we end up with: $javac .\SimpleInterface.java .\SimpleInterface.java:18: error: SimpleInterfaceImpl is not abstract and does not override abstract method doSomeOtherWork() in SimpleInterface class SimpleInterfaceImpl implements SimpleInterface{ ^ 1 error And this limitation makes it almost impossible to extend/improve the existing interfaces and APIs. The same challenge was faced while enhancing the Collections API in Java 8 to support lambda expressions in the API. To overcome this limitation a new concept is introduced in Java 8 called default methods which is also referred to as Defender Methods or Virtual extension methods. Default methods are those methods which have some default implementation and helps in evolving the interfaces without breaking the existing code. Lets look at an example: public interface SimpleInterface { public void doSomeWork(); //A default method in the interface created using "default" keyword default public void doSomeOtherWork(){ System.out.println("DoSomeOtherWork implementation in the interface"); } } class SimpleInterfaceImpl implements SimpleInterface{ @Override public void doSomeWork() { System.out.println("Do Some Work implementation in the class"); } /* * Not required to override to provide an implementation * for doSomeOtherWork. */ public static void main(String[] args) { SimpleInterfaceImpl simpObj = new SimpleInterfaceImpl(); simpObj.doSomeWork(); simpObj.doSomeOtherWork(); } } and the output is: Do Some Work implementation in the class DoSomeOtherWork implementation in the interface This is a very brief introduction to default methods. One can read in depth about default methods here.

Lambda expressions are coming to Java 8 and together with Raoul-Gabriel Urma and Alan Mycroft I started writing a book on this topic.

We all have extensively used Collection classes like List, Map and their derived versions. And each time we used them we had to iterate through them to either find some element or update the elements or find different elements matching some condition. Consider a List of Person as shown below: List personList = new ArrayList<>(); personList.add(new Person("Virat", "Kohli",22)); personList.add(new Person("Arun", "Kumar",25)); personList.add(new Person("Rajesh", "Mohan", 32)); personList.add(new Person("Rahul", "Dravid", 35)); To find out all the Person instances with age greater than 30, we would do: List olderThan30OldWay = new ArrayList<>(); for ( Person p : personList){ if ( p.age >= 30){ olderThan30OldWay.add(p); } } System.out.println(olderThan30OldWay); and this gives me the output as: [Rajesh Mohan, 32, Rahul Dravid, 35] The code is easy to write, but is it not a bit more verbose, especially the iteration part? Why would we have to iterate? Would it not be cool if there was an API which would iterate the contents and give us the end result i.e we give the source List and use a series of method calls to get us the result List we are looking for? Yes, this is possible in other languages like Scala, Groovy which support passing closures and also support internal iteration. But is there a solution for Java developers? Yes, this exact problem is being solved by introducing support for Lambda Expressions(closures) and a enhanced Collection API to leverage the lambda expression support. The sad news is that it’s going to be part of Java 8 and will take some time to be into mainstream development. Leveraging the Java 8 enhancements to the above scenario As I said before the Collections API is being enhanced to support the use of Lambda Expression and more about it can be read here. Instead of adding all the new APIs to the Collection class the JDK team created a new concept called “Stream” and added most of the APIs in that class. “Stream” is a sequence of elements obtained from the Collection from which it is created. To read more about the origin of Stream class please refer to this document. To implement the example I started with using the enhancements in Java 8 we would be using few new APIs namely: stream(), filter(), collect(), Collectors.toCollection(). stream(): Uses the collection on which this API is called to create an instance of Stream class. filter():This method accepts a lambda expression which takes in one parameter and returns a boolean value. This lambda expression is written as a replacement for implementing the Predicate class. collect(): There are 2 overloaded versions of this method. The one I am using here takes an instance of Collector. This method takes the contents of the stream and constructs another collection. This construction logic is defined by the Collector. Collectors.toCollection(): Collectors is a factory for Collector. And the toCollection() takes a Lambda expression/Method reference which should return a new instance of any derivatives of Collection class. With brief introduction to the APIs used, let me show the code which is equivalent to the first code sample: List olderThan30 = //Create a Stream from the personList personList.stream(). //filter the element to select only those with age >= 30 filter(p -> p.age >= 30). //put those filtered elements into a new List. collect(Collectors.toCollection(() -> new ArrayList())); System.out.println(olderThan30); The above code uses both Internal iteration and lambda expressions to make it intuitive, concise and soothing to the eye. If you are not familiar with the idea of Lambda Expressions, check out my previous entry which covers in brief about Lambda expressions.

This article will provide the complete root cause analysis details and resolution of a Java heap memory leak (Apache OpenJPA leak) affecting an Oracle Weblogic server 10.0 production environment. This post will also demonstrate the importance to follow the Java Persistence API best practices when managing the javax.persistence.EntityManagerFactory lifecycle. Environment specifications Java EE server: Oracle Weblogic Portal 10.0 OS: Solaris 10 JDK: Oracle/Sun HotSpot JVM 1.5 32-bit @2 GB capacity Java Persistence API: Apache OpenJPA 1.0.x (JPA 1.0 specifications) RDBMS: Oracle 10g Platform type: Web Portal Troubleshooting tools Quest Foglight for Java (Java heap monitoring) MAT (Java heap dump analysis) Problem description & observations The problem was initially reported by our Weblogic production support team following production outages. An initial root cause analysis exercise did reveal the following facts and observations: Production outages were observed on regular basis after ~2 weeks of traffic. The failures were due to Java heap (OldGen) depletion e.g. OutOfMemoryError: Java heap space error found in the Weblogic logs. A Java heap memory leak was confirmed after reviewing the Java heap OldGen space utilization over time from Foglight monitoring tool along with the Java verbose GC historical data. Following the discovery of the above problems, the decision was taken to move to the next phase of the RCA and perform a JVM heap dump analysis of the affected Weblogic (JVM) instances. JVM heap dump analysis ** A video explaining the following JVM Heap Dump analysis is now available here. In order to generate a JVM heap dump, the supported team did use the HotSpot 1.5 jmap utility which generated a heap dump file (heap.bin) of about ~1.5 GB. The heap dump file was then analyzed using the Eclipse Memory Analyzer Tool. Now let’s review the heap dump analysis so we can understand the source of the OldGen memory leak. MAT provides an initial Leak Suspects report which can be very useful to highlight your high memory contributors. For our problem case, MAT was able to identify a leak suspect contributing to almost 600 MB or 40% of the total OldGen space capacity. At this point we found one instance of java.util.LinkedList using almost 600 MB of memory and loaded to one of our application parent class loader (@ 0x7e12b708). The next step was to understand the leaking objects along with the source of retention. MAT allows you to inspect any class loader instance of your application, providing you with capabilities to inspect the loaded classes & instances. Simply search for the desired object by providing the address e.g. 0x7e12b708 and then inspect the loaded classes & instances by selecting List Objects > with outgoing references. As you can see from the above snapshot, the analysis was quite revealing. What we found was one instance of org.apache.openjpa.enhance.PCRegistry at the source of the memory retention; more precisely the culprit was the _listeners field implemented as a LinkedList. For your reference, the Apache OpenJPA PCRegistry is used internally to track the registered persistence-capable classes. Find below a snippet of the PCRegistry source code from Apache OpenJPA version 1.0.4 exposing the _listeners field. /** * Tracks registered persistence-capable classes. * * @since 0.4.0 * @author Abe White */ publicclass PCRegistry { // DO NOT ADD ADDITIONAL DEPENDENCIES TO THIS CLASS privatestaticfinal Localizer _loc = Localizer.forPackage (PCRegistry.class); // map of pc classes to meta structs; weak so the VM can GC classes privatestaticfinal Map _metas = new ConcurrentReferenceHashMap (ReferenceMap.WEAK, ReferenceMap.HARD); // register class listeners privatestaticfinal Collection _listeners = new LinkedList(); …………………………………………………………………………………… Now the question is why is the memory footprint of this internal data structure so big and potentially leaking over time? The next step was to deep dive into the _listeners LinkedLink instance in order to review the leaking objects. We finally found that the leaking objects were actually the JDBC & SQL mapping definitions (metadata) used by our application in order to execute various queries against our Oracle database. A review of the JPA specifications, OpenJPA documentation and source did confirm that the root cause was associated with a wrong usage of the javax.persistence.EntityManagerFactory such of lack of closure of a newly created EntityManagerFactory instance. If you look closely at the above code snapshot, you will realize that the close() method is indeed responsible to cleanup any recently used metadata repository instance. It did also raise another concern, why are we creating such Factory instances over and over… The next step of the investigation was to perform a code walkthrough of our application code, especially around the life cycle management of the JPA EntityManagerFactory and EntityManager objects. Root cause and solution A code walkthrough of the application code did reveal that the application was creating a new instance of EntityManagerFactory on each single request and not closing it properly. public class Application { @Resource private UserTransaction utx = null; // Initialized on each application request and not closed! @PersistenceUnit(unitName = "UnitName") private EntityManagerFactory emf = Persistence.createEntityManagerFactory("PersistenceUnit"); public EntityManager getEntityManager() { return this.emf.createEntityManager(); } public void businessMethod() { // Create a new EntityManager instance via from the newly created EntityManagerFactory instance // Do something... // Close the EntityManager instance } } This code defect and improver use of JPA EntityManagerFactory was causing a leak or accumulation of metadata repository instances within the OpenJPA _listeners data structure demonstrated from the earlier JVM heap dump analysis. The solution of the problem was to centralize the management & life cycle of the thread safe javax.persistence.EntityManagerFactory via the Singleton pattern. The final solution was implemented as per below: Create and maintain only one static instance of javax.persistence.EntityManagerFactory per application class loader and implemented via the Singleton Pattern. Create and dispose new instances of EntityManager for each application request. Please review this discussion from Stackoverflow as the solution we implemented is quite similar. Following the implementation of the solution to our production environment, no more Java heap OldGen memory leak is observed. Please feel free to provide your comments and share your experience on the same.

As part of JSR 335Lambda expressions are being introduced to the Java language from Java 8 onwards and this is a major change in the Java language.

This Post explains Topics in Active MQ (Message Broker) with Subscribing and Publishing. For this we will write two java clients. As we did for wso2 Message Broker TopicSubscriber.java to Subcribe for messages TopicPublisher.java to to Publish the messages Let's Start. [1] Get Active MQ from http://activemq.apache.org/download.html [1.1] Start Active MQ from \bin\activemq.bat You can see the started server form http://localhost:8161/admin/ [2] Create Porject "Client" on IDE that you preferred [3] Add activemq-all-5.7.0.jar to lib Dir in the project (activemq-all-5.7.0.jar can be found in root folder) [4] Creat class "TopicSubscriber.java" to Subcribe for messages package simple; import java.util.Properties; import javax.jms.JMSException; import javax.jms.Message; import javax.jms.MessageListener; import javax.jms.Session; import javax.jms.TextMessage; import javax.jms.Topic; import javax.jms.TopicConnection; import javax.jms.TopicConnectionFactory; import javax.jms.TopicSession; import javax.naming.InitialContext; import javax.naming.NamingException; public class TopicSubscriber { private String topicName = "news.sport"; private String initialContextFactory = "" +"org.apache.activemq.jndi.ActiveMQInitialContextFactory"; private String connectionString = "tcp://" +"localhost:61616"; private boolean messageReceived = false; public static void main(String[] args) { TopicSubscriber subscriber = new TopicSubscriber(); subscriber.subscribeWithTopicLookup(); } public void subscribeWithTopicLookup() { Properties properties = new Properties(); TopicConnection topicConnection = null; properties.put("java.naming.factory.initial", initialContextFactory); properties.put("connectionfactory.QueueConnectionFactory", connectionString); properties.put("topic." + topicName, topicName); try { InitialContext ctx = new InitialContext(properties); TopicConnectionFactory topicConnectionFactory = (TopicConnectionFactory) ctx .lookup("QueueConnectionFactory"); topicConnection = topicConnectionFactory.createTopicConnection(); System.out .println("Create Topic Connection for Topic " + topicName); while (!messageReceived) { try { TopicSession topicSession = topicConnection .createTopicSession(false, Session.AUTO_ACKNOWLEDGE); Topic topic = (Topic) ctx.lookup(topicName); // start the connection topicConnection.start(); // create a topic subscriber javax.jms.TopicSubscriber topicSubscriber = topicSession .createSubscriber(topic); TestMessageListener messageListener = new TestMessageListener(); topicSubscriber.setMessageListener(messageListener); Thread.sleep(5000); topicSubscriber.close(); topicSession.close(); } catch (JMSException e) { e.printStackTrace(); } catch (NamingException e) { e.printStackTrace(); } catch (InterruptedException e) { e.printStackTrace(); } } } catch (NamingException e) { throw new RuntimeException("Error in initial context lookup", e); } catch (JMSException e) { throw new RuntimeException("Error in JMS operations", e); } finally { if (topicConnection != null) { try { topicConnection.close(); } catch (JMSException e) { throw new RuntimeException( "Error in closing topic connection", e); } } } } public class TestMessageListener implements MessageListener { public void onMessage(Message message) { try { System.out.println("Got the Message : " + ((TextMessage) message).getText()); messageReceived = true; } catch (JMSException e) { e.printStackTrace(); } } } } [5] Creat class "TopicPublisher.java" to to Publish the messages package simple; import javax.jms.*; import javax.naming.InitialContext; import javax.naming.NamingException; import java.util.Properties; public class TopicPublisher { private String topicName = "news.sport"; private String initialContextFactory = "org.apache.activemq" +".jndi.ActiveMQInitialContextFactory"; private String connectionString = "tcp://localhost:61616"; public static void main(String[] args) { TopicPublisher publisher = new TopicPublisher(); publisher.publishWithTopicLookup(); } public void publishWithTopicLookup() { Properties properties = new Properties(); TopicConnection topicConnection = null; properties.put("java.naming.factory.initial", initialContextFactory); properties.put("connectionfactory.QueueConnectionFactory", connectionString); properties.put("topic." + topicName, topicName); try { // initialize // the required connection factories InitialContext ctx = new InitialContext(properties); TopicConnectionFactory topicConnectionFactory = (TopicConnectionFactory) ctx .lookup("QueueConnectionFactory"); topicConnection = topicConnectionFactory.createTopicConnection(); try { TopicSession topicSession = topicConnection.createTopicSession( false, Session.AUTO_ACKNOWLEDGE); // create or use the topic System.out.println("Use the Topic " + topicName); Topic topic = (Topic) ctx.lookup(topicName); javax.jms.TopicPublisher topicPublisher = topicSession .createPublisher(topic); String msg = "Hi, I am Test Message"; TextMessage textMessage = topicSession.createTextMessage(msg); topicPublisher.publish(textMessage); System.out.println("Publishing message " +textMessage); topicPublisher.close(); topicSession.close(); Thread.sleep(20); } catch (InterruptedException e) { e.printStackTrace(); } } catch (JMSException e) { throw new RuntimeException("Error in JMS operations", e); } catch (NamingException e) { throw new RuntimeException("Error in initial context lookup", e); } } } [6] Firstly Run "TopicSubscriber.java" and then run "TopicPublisher.java" Here is out put from both TopicSubscriber:: Create Topic Connection for Topic news.sport Got the Message : Hi, I am Test Message TopicPublisher:: Use the Topic news.sport Publishing message ActiveMQTextMessage {commandId = 0, responseRequired = false, messageId = ID:Madhuka-THINK-51683-1359787878456-1:1:1:1:1, originalDestination = null, originalTransactionId = null, producerId = null, destination = topic://news.sport, transactionId = null, expiration = 0, timestamp = 1359787878729, arrival = 0, brokerInTime = 0, brokerOutTime = 0, correlationId = null, replyTo = null, persistent = true, type = null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null, compressed = false, userID = null, content = null, marshalledProperties = null, dataStructure = null, redeliveryCounter = 0, size = 0, properties = null, readOnlyProperties = false, readOnlyBody = false, droppable = false, text = Hi, I am Test Message} [More] Here is full message that we have send to TopicSubscriber. We can get that any parameter in above. Here is sample to get TimeStamp and ID from JMS message. public class TestMessageListener implements MessageListener { public void onMessage(Message message) { try { System.out.println("Got the Message TimeStamp: " + message.getJMSTimestamp()); System.out.println("Got the Message JMS ID : " + message.getJMSMessageID()); messageReceived = true; } catch (JMSException e) { e.printStackTrace(); } } } Now go to ActiveMQ server at http://localhost:8161/admin/ See that Topic and message count for that topic. Now you time to check more in 'Active MQ'

We are welcoming some new colleagues that come from a Java background in the Onebip team, both from the development and operations field. Here's a primer on learning PHP in this situation, that you may find useful when introducing similar people in your PHP-based projects. The absolute basics Before being able to discuss meaningful matters over PHP code, these pages from the manual should be sent to each interested developer. What is the basic syntax and the available data types of PHP. Primitive types are diffused like in Java, but there's no autoboxing as there is no object equivalent for them. Strings are immutable and one of the most important types. They can be defined with single or double quotes, and their manipulation functions follow the C api. Arrays are the glue of the language, working as lists, sets and maps while wrapped into objects. They can always be traversed with foreach(), and keep an eye on the available functions to avoid rewriting array_search() or sort() by accident. Operators work differently on primitive values and on objects: == is different from === in both these context, but in two ways. Other everyday operators are `.` and `+=`. PHP's object paradigm is borrowed from Java: it supports concrete and abstract classes, interfaces, and the private, protected and public scopes. Type hints (which are actually not hints but strong preconditions) are what is most similar to Java static type safety mechanisms. I personally strongly favors their usage. Namespaces are packages, use statements are imports. However, you don't always have to write them. The deployment model of PHP consists of an interpreter instantiated N times, running on many shared nothing processes. Don't care about: The installation process of Apache and PHP: if you work in a team, you will get good help on that, and you're going to do this only once per project probably. For example, we provide a virtual machine ready for development. The function syntax is also not very useful by itself in 2013, skip directly to objects and their methods. Take a look at anonymous functions, however. Exceptions work mostly as in Java, so don't bother reading about them before coding. Sessions are also evil in web services nowadays, so ignore anything that start with $_SESSION or session_*(): write shared nothing services (that may be restricted to our team and projects.) In general, ignore also low-level APIs like setcookie() or exec() if you already have an higher-level abstraction in the application you're working on, being this abstraction a library or your own code. It's important to know how cookies are transmitted according to HTTP, but coming from another language you already know the protocol. So you write objects that go into a graph At least that's how I see programming these days. However, you have to exit the process sometimes, and PHP provides several APIs for that. The database APIs such as the mongo and PDO extensions, working respectively with MongoDB and relational database. However, if you already use a database abstraction layer, learn just that as there is nothing interesting to see at this lower level. The DateTime object and its cousins, at least know how to call its constructor and the format() method. You should know these APIs exist in order to look them up on demand: json_*() functions, the SimpleXMLElement class, the mcrypt extension and hash_hmac(), mail(). They're just a click away at php.net/function_or_class_name, and I think these names are pretty self-explanatory to tell you when you should read about them. Watch out for: Some topics are always controversial, and get confusing for PHP beginners. Raise your alert level when you feel you're encountering some strange behavior. the difference between == and === comes out often, and it cannot be reduced to "just use === by default" since all primitive types coming from HTTP requests in the classic x-www-form-urlencoded Content-Type are usually strings. php.ini settings may change the output of your application and the actual flow of execution depending on error reporting levels and display options. One never ceases to learn, so when you encounter some problematic directive, take a quick read of the rest of the directives for that extension. Since some people believe web development is the concatenation of strings (it is not), it is tempting to reinvent the wheel; for example, writing functions for composing URLS from paths and query strings. http_build_query() solves that problem as other Composer-ready packages do, so take a look around before starting to implement commodity algorithms yourself. Stack Overflow and the PHP manual itself are good places to start if your problem has been already solved by someone else in the last 10 years. Perform a kata To test your level of proficiency with PHP, execute a small kata with TDD which incidentally will also check your usage of PHPUnit. For example, classic katas are: The Fizz Buzz, maybe with a web interface. The Roman numbers kata: write a function that transforms 42 in XLII. The Game of Life, although it is probably too complex for your first PHP project if you didn't already solve it in another language.

Introduction The IT buzzword of 2012 is without a doubt Big Data. It’s new and here to stay, and for a good reason. Big data is data that exceeds the processing capacity of conventional database systems. Great examples are CERN with the Large Hadron Collider, whose experiments generate 25 petabytes of data annually, or Walmart, which handles more than one million customer transaction every hour. Problems These vast amounts of data leave us with two problems. Problem 1: To gain value from this data, one must choose an alternative way to process it. The value of big data to an organization falls into two categories: analytical use, and enabling new products. Big data analytics can reveal insights hidden previously by data too costly to process, such as peer influence among customers, revealed by analyzing shoppers’ transactions, social and geographical data. Being able to process every item of data in reasonable time removes the troublesome need for sampling and promotes an investigative approach to data, in contrast to the somewhat static nature of running predetermined reports. Problem 2: The data is too big, moves too fast, or doesn’t fit the strictures of your database architectures. Remember the CERN case where the LHC produces over 25 Petabytes of data annually? No “classic” database architecture or setup is capable of holding these amounts of data. Solutions Fortunately, both problems can be solved by implementing the correct infrastructure and rethinking data storage. There are two critical factors in Big Data environments: size and speed. We already discussed the vast amounts of data and desire to be able to access and process the data fast. The latter is the main differentiator from more traditional data warehouses. Just imagine what you can do when you can access all your data real-time. Enter big data. A common Big Data implementation is an in-memory data grid that lives in a distributed cluster, ensuring both speed, by storing data in-memory, and capacity by using scalability features provided by a cluster. As a bonus, availability is ensured by using a distributed cluster. As for the data storage, there are typically two kinds: in-memory databases and in-memory data grids. But first some background. It is not a new attempt to use main memory as a storage area instead of a disk. In our daily lives there are numerous examples of main memory databases (MMDB), as they perform much faster than disk-based databases. An every day example is a mobile phone. When you SMS or call someone most mobile service providers use MMDB to get the information on your contact as soon as possible. The same applies to your phone. When someone calls you, the caller details are looked up in the contacts application, usually providing a name and sometimes a picture. In memory data grids In Memory Data Grid (IMDG) is the same as MMDB in that it stores data in main memory, but it has a totally different architecture. The features of IMDG can be summarized as follows: Data is distributed and stored on multiple servers. Each server operates in the active mode. A data model is usually object-oriented (serialized) and non-relational. According to the necessity, you often need to add or reduce servers. No traditional database features such as tables. In other words, IMDG is designed to store data in main memory, ensure scalability and store an object itself. These days, there are many IMDG products, both commercial and open source. Some of the most commonly used products are: Hazelcast (http://www.hazelcast.com) JBoss Infinispan (http://www.jboss.org/infinispan) GridGain DataGrid (http://www.gridgain.com/features/in-memory-data-grid/) VMware Gemfire (http://www.vmware.com/nl/products/application-platform/vfabric-gemfire/overview.html) Oracle Coherence (http://www.oracle.com/technetwork/middleware/coherence/overview/index.html) Gigaspaces XAP (http://www.gigaspaces.com/datagrid) Terracotta Enterprise Suite (http://terracotta.org/products/enterprise-suite) Why Memory? The main reasons for using main memory for data storage are once again the two main themes of Big Data: speed and capacity. The processing performance of main memory is 800 times faster than an HDD and up to 40 times faster than an. Moreover, the latest x86 server supports main memory of hundreds of GB per server. It is said that the limit of a traditional processing database’s (OLTP) data capacity is approximately 1 TB and that the OLTP processing data capacity would not increment well. If servers using main memory of 1 TB or larger become more commonly used, you will be able to conduct operations with the entire data placed in main memory, at least in the field of OLTP. IMDG Architecture To use main memory as a storage area, two weak points should be overcome: Limited capacity: involves data that exceeds the maximum capacity of the main memory of the server Reliability: involves data loss in case of a (system) failure. IMDG overcomes the limit of capacity by ensuring horizontal scalability using a distributed architecture, and resolves the issue of reliability through a replication system as part of the grid (or a distributed cluster). Now let’s discuss how an IMDG actually works. First of all, it is important to understand that an IMDG is not the same as an in-memory database, also referred to as MMDB (main memory databases). Typical examples of MMDBs are Oracle TimesTen or Sap Hana. MMDBs are full database products that simply reside in memory. As a result of being a full-blown database, they also carry the weight and overhead of database management features. IMDG is different. No tables, indexes, triggers, stored procedures, process managers etc. Just plain storage. The data model used in IMDG is key-value pairs. A key-value pair is a list with only two parts: a key and a value. The key can be used for storing and retrieving the values in the list. A key can be compared to the index or primary key of a table in a database. Note that IMDG are closely tied to development environments such as Java as the key-value pairs are represented by the structures provided by such a programming environment. Most IMDGs are written in Java, and can only be used within other Java applications. Therefore, the values of key-value pairs can be anything supported by Java, ranging from simple data types such as a string or number, to complex objects. This overcomes the two important hurdles: as you can store complex Java objects as value, there’s no need to translate these objects into a relational datamodel (which is the case in more traditional applications using a database for storage). Furthermore, the seeming limitation of being able to store only one value per key, is actually no limitation at all. Large memory sizes Most of the products introduced above use Java as an implementation language. Java reserves and uses a part of the RAM (internal memory) for dynamic memory allocation. This reserved memory space is called the Java heap. All runtime objects created by a Java application are stored in heap. Using large amounts of data causes two problems. Size limitation: By default, the heap size is 128 MB, but for current business applications, this limit is reached easily. Once the heap is “full”, no new objects can be created and the Java application will show some nasty errors. Performance: It is possible to increase the size of the heap, but this introduces some new problems. When a heap reaches a size of more than 4 gigabytes, Java will have serious issues with memory managements, causing your application to slow down or even freeze. Java has a feature called Garbage Collector, which periodically scans the heap and checks each object if it is still valid and being used. If not, the garbage collector removes the object and defragments the newly available space. The problem is, the larger the heap size, the more work to do for the garbage collector, resulting in performance degradation. Imagine a large bank has a Java application that manages customers, accounts and transactions. We have seen that an IMDG allows the application to store and access all data very quickly by caching it in memory, instead of storing the data in relatively slow databases. Let’s assume the combined data has a size of 40 gigabytes. Storing it in heap is simply not possible, considering the performance penalties of Java’s memory management capabilities. The graph below illustrates the garbage collection pause time when placing cached data in heap: Terracotta’s BigMemory product has a method to overcome these limitations. The method is to use an off-heap memory (direct buffer). Data will not be stored in Java’s heap, but directly in the available internal memory (RAM). Since, this is not subject to Java’s garbage collector, there are no performance penalties. The differences on performance are significant, as can be seen in the graph below: Using off-heap storage has some major benefits: You can use all the available memory on your machine, not just the memory that is allocated to the heap (usually less that 512 Mb). This allows you to store more data in a in-memory data grid, greatly speeding up your application. The heap can be relieved by storing data in native memory, speeding up Java applications as less heap space has to be garbage collected. Clustering, fail over and high availability So far, we have seen IMDG features that are applicable to a single server. However, the real power of IMDG lies in it’s networking and clustering capabilities, providing features as data replication, data synchronization between clients, fail over and high availability. To achieve this, a cluster of servers (or server array) acts a backbone of the infrastructure. Applications (that still can have their own IMDG or off-heap cache) that are connected to the cluster can share, replicate and backup their data with either the cluster or other applications. The graph below depicts a typical setup using Terracotta's BigMemory: The caches on the application servers are usually referred to as “level 1” cache, while the data cache on the server array is referred to as “level 2” cache. There are many different scenarios possible for storing, clustering, synchronizing and replicating data. Covering all these topics goes far beyond the scope of this article. For more information, consult the technical documentation of the product of your choice. Conclusion Big Data brings us some new challenges. First of all, storing and accessing vast amounts of data makes us rethink traditional methods and technologies. Next, there’s the question what to do with all the available data. The potential value for marketing, financial and other businesses is huge. In order to facilitate Big Data, in-memory data grids are considered the best option. IMDGs with off-heap storage are even more powerful, allowing data centric enterprise application to overcome certain limits of the Java platform, such as memory and performance constraints. As the amount of data that (large) companies produce and store, grows exponentially, databases will hit a limit. Accessing your data without a performance penalty simply will not be possible. The answer to this is using an IMDG.

In this part, I will be briefing about different types of Database Normalizations using a sample data model. What is Database Normalization? Normalization is the process of efficiently organizing data in the database. Primary Goal of Normalization? Eliminating redundant data & ensuring meaningful data dependencies. Types of Normalization The following are the three most common normal forms in the database normalization process First Normal Form (1NF) Second Normal Form (2NF) Third Normal Form (3NF) Sample Data Model for Demonstration The following data model will be used to demonstrate all the three normal forms First Normal Form (1NF) First Normal Form (1NF) sets the very basic rules for an organized database: Create separate set of tables for each group of related data and identify each row with a unique columns [primary key] or set of columns [composite key] Eliminate duplicate columns from the table The following data model depicts the tables after 1NF rules are applied - Second Normal Form (2NF) Second Normal Form (2NF) further addresses the concept of removing duplicate data: Meet all the requirements of the first normal form Remove subsets of data that apply to multiple rows of a table and place them in separate tables Create relationships between these new tables and their predecessors through the use of foreign keys So basically the objective of the Second Normal Form is to take that is only partly dependent on the primary key and enter that data into another table. The following data model depicts the tables after 2NF rules are applied. Data from EMPLOYEE_TABLE is split into 2 tables – EMPLOYEE_TABLE and EMPLOYEE_HR_TABLE. Similarly data from CUSTOMER_TABLE is moved to CUSTOMER_TABLE and CUSTOMER_ORDER table Third Normal Form (3NF) Third normal form (3NF) goes one large step further: Meet all the requirements of the second normal form. Remove columns that are not dependent upon the primary key. The following data model depicts the tables after 3NF rules are applied. Further state and country details are moved to their own tables because they are not dependent on the primary key. Advantages of Normalizing the Database There are several advantages of normalization - Data can be stored as small atomic pieces Saves space Increases speed Reduces data anomalies Easy maintenance Other parts of this series include: Part 1 – ACID Properties Part 2 – Keys Part 4 – Database Transactions [coming soon] Part 5 – Indexes [coming soon]

Need help dealing with Maven's non-resolvable parent problem? Check out this post to learn how.

There are currently two schools of thought in developing Web Services – one being the standards-based traditional approach [ SOAP ] and the other, simpler school of thought [ REST ]. This article quickly compares one with the other - REST SOAP Assumes a point-to-point communication model–not usable for distributed computing environment where message may go through one or more intermediaries Designed to handle distributed computing environments Minimal tooling/middleware is necessary. Only HTTP support is required Requires significant tooling/middleware support URL typically references the resource being accessed/deleted/updated The content of the message typically decides the operation e.g. doc-literal services Not reliable – HTTP DELETE can return OK status even if a resource is not deleted Reliable Formal description standards not in widespread use. WSDL 1.2, WADL are candidates. Well defined mechanism for describing the interface e.g. WSDL+XSD, WS-Policy Better suited for point-to-point or where the intermediary does not play a significant role Well suited for intermediated services No constraints on the payload Payload must comply with the SOAP schema Only the most well established standards apply e.g. HTTP, SSL. No established standards for other aspects. DELETE and PUT methods often disabled by firewalls, leads to security complexity. A large number of supporting standards for security, reliability, transactions. Built-in error handling (faults) No error handling Tied to the HTTP transport model Both SMTP and HTTP are valid application layer protocols used as Transport for SOAP Less verbose More verbose

It is common that most of the developers use their own logging frameworks at the time of development and that force organizations to maintain configuration for each logging framework. Normally switching from logging level from DEBUG to INFO sometimes requires a application restart in production. SLF4J is the latest logging facade helps to plug-in desired logging framework at deployment time. The article further talks about usage of the SLF4J with logback. SLF4J - Simple Logging Facade for Java API helps to plug-in desired logging implementation at deployment time. Logback - helps to change logging configuration through JMX at runtime with out restarting your applications in production. I hope this article helps to get high level overview of SLF4J/Logback and migrating your existing apps to common logging approach. SLF4J This is simple logging façade to abstract the various logging frameworks such as logback, log4j, commons-logging and default java logging implementation (java.util.logging). This primarily enables the user to inlcude desired logging framework at deployment time. It is lightweight and nearly adds a zero overhead on performance. Note that SLF4j doesn’t replace any logging framework; it is just a façade around any standard logging framework. If slf4j doesn’t find any logging framwork in classpath, by default it prints the logs in console. Logback This is an improved version of log4j and natively supports the slf4j, hence migrating from other logging frameworks such as log4j and java.util.logging is quite possible. Since the logback natively supports slf4j, the combination of using slf4j with this framework is relatively faster than the slf4j with other logging frameworks. Logging configuration can be done either in xml or groovy. *One important feature is that it exposes the configuration through JMX hence configuration (debug to info etc) can be changed via JMX console with out restarting the application. Also, it does print the artifact version as part of the exception stacktrace that may be helpful for debugging. java.lang.NullPointerException: null at com.fimt.poc.LoggingSample.(LoggingSample.java:16) [classes/:na] at com.fimt.poc.LoggingSample.main(LoggingSample.java:23) [fimt-logging-poc-1.0.jar/:1.0] **Reasons to prefer logback over log4j is well explainedhere SLF4J api usage in java classes (1) Import the Logger and LoggerFactory from org.slf4j package import org.slf4j.Logger; import org.slf4j.LoggerFactory; (2) Declare the logger class as, private final Logger logger = LoggerFactory.getLogger(LoggingSample.class); (3) Use debug, warn, info, error and trace with appropriate parameters. All methods by default takes the string as an input. logger.info("This is sample info statement"); SLF4J with Logback Include the following dependency pom.xml, it pulls its depedencies logback-core and slf4j-api in addition to logback-classic ch.qos.logback logback-classic 1.0.7 SLF4J can be used with existing logging framworks log4j, common-logging and java.util.logging (JUL). Required dependencies are mentioned below. SLF4J with Log4j Include the following dependency pom.xml, it pulls its depedencies log4jand slf4j-api in addition to slf4j-log4j12 artifact. org.slf4j slf4j-log4j12 1.7.2 SLF4J with JUL (java.util.logging) Include the following dependency pom.xml, it pulls its depedency slf4j-api in addition to slf4j-jdk14 artifact. org.slf4j slf4j-jdk14 1.7.2 Migrating existing projects logging to logback framework Step: 1 – Update existing project pom.xml Add the right dependency mentioned above. Also remove the unused log4j/commons logging dependencies. Step: 2 – Update java files with SLF4J API Scan all java files and replace log4j or java.util.logging classes in to SLF4J api classes. This can be done using the tool java -jar slf4j-migrator-1.7.2.jar . Detailed documentation and limitation is mentioned here This tool replaces the logging apis of log4j, commons-logging and java.util.logging in to SLF4J API classes. Step: 3 – Convert log4j.properties to logback.xml Logback provides a online translator which converts log4j properties in to logback.xml File Appenders: Like other logging frameworks, logback implementation also supports various file appenders. Daily file rolling: logFile.%d{yyyy-MM-dd}.log 30 Roll log files based on size: tests.%i.log.zip 1 10 5MB Layout %-4relative [%thread] %-5level %logger{35} - %msg%n