Wednesday, July 22, 2015

When Serializable and Externalizable appear in the same class hierarchy

In my post Difference between Serializable and Externalizable, we know that a Java class can choose the object serialization mechanism by either implementing java.io.Serializable or  java.io.Externalizable. The question is what is the behavior if some of the parent classes of an object implement Serializable and some other parent classes implement Externalizable?

Given the class diagram for Programmer as below:


And the implementation for

Person 

import java.io.Serializable;

public class Person implements Serializable {
    int age;
}   

Employee

import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;

public class Employee extends Person implements Externalizable {
    int salary;
    
    @Override
    public void writeExternal(ObjectOutput out) throws IOException {
        out.writeInt(salary);
    }

    @Override
    public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
        salary = in.readInt();
    }
}

Programmer

import java.io.Serializable;

public class Programmer extends Employee implements Serializable {
    int yearsOfExp;
}

In the program below, we create a Programmer instance and assign a value to each of its fields. Note that neither of its fields is transient or static, they are all eligible for serialization. When we execute the program below, guess what is the result.

public class App {
    public static void main(String[] args) throws Exception {
        Programmer programmer = new Programmer();
        programmer.age = 30;
        programmer.salary = 6000;
        programmer.yearsOfExp = 8;
        
        serialize(programmer, "programmer.ser");
        
        programmer = (Programmer) deserialize("programmer.ser");
        System.out.println(programmer.age 
                + " " + programmer.salary 
                + " " + programmer.yearsOfExp);
    }
    
    public static void serialize(Object obj, String fileName)
            throws IOException {
        try (ObjectOutputStream out 
                = new ObjectOutputStream(new FileOutputStream(fileName))) {
            out.writeObject(obj);
        }
    }

    public static Object deserialize(String fileName)
            throws IOException, ClassNotFoundException {
        try (ObjectInputStream in 
                = new ObjectInputStream(new FileInputStream(fileName))) {
            Object obj = in.readObject();
            return obj;
        }
    }
}

Result:
0 6000 0
Only the salary get serialized.

Answer

The existence of Externalizable in any level of the class hierarchy will supersedes Serializable. This means that serialization runtime will call the overrode Externalizable read/write method instead of automatically serialize/deserialize the target object's fields.

So what can we do if we want to serialize all Programmer's fields?

As Programmer extends Employee, it is Externalizable (Programmer to implements Serializable is redundant). The Programmer could override the Externalizable methods to read/write age and yearsOfExp.

public class Programmer extends Employee {
    int yearsOfExp;
    
    @Override
    public void writeExternal(ObjectOutput out) throws IOException {
        super.writeExternal(out);
        out.writeInt(age);
        out.writeInt(yearsOfExp);
    }

    @Override
    public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
        super.readExternal(in);
        age = in.readInt();
        yearsOfExp = in.readInt();
    }
}

Re-run the App program again, and this time we will get the desired result as below:
30 6000 8
In short, whenever an Externalizable class appears in the class hierarchy, serialization runtime is expecting the programmer to handle the field serialization by him/herself.

Tuesday, July 21, 2015

Java NIO FileChannel

Creating FileChannel instance

java.nio.FileChannel is a new way of transferring file data. In traditional I/O, file data is read via java.io.FileInputStream and written via java.io.FileOutputStream. A FileChannel instance supports read and write data from or to a file. However, not all the FileChannel instances support both operations by default. The construction of an instance of FileChannel determines its reading and writing capability.

Create a FileChannel instance from FileInputStream instance

A FileChannel instance that is created via getChannel() method of a java.io.FileInputStream instance can only read data from a file into a buffer. An attempt to write data into this channel will throw java.nio.channels.NonWritableChannelException.

FileInputStream is = new FileInputStream("D:\\test.txt");
FileChannel channel = is.getChannel();
channel.read(ByteBuffer.allocate(10)); //OK
channel.write(ByteBuffer.allocate(10)); //java.nio.channels.NonWritableChannelException

Create a FileChannel instance from FileOutputStream instance

A FileChannel instance that is created via getChannel() method of a java.io.FileOutputStream instance can only write data from a buffer into a file. An attempt to read data from this channel will throw java.nio.channels.NonReadableChannelException.

FileOutputStream os = new FileOutputStream("D:\\test.txt");
FileChannel channel = os.getChannel();
channel.write(ByteBuffer.allocate(10)); //OK
channel.read(ByteBuffer.allocate(10)); //java.nio.channels.NonReadableChannelException

Create a FileChannel instance from RandomAccessFile instance

Creation of a java.io.RandomAccessFile instance requires the mode input. A FileChannel instance that is created via getChannel() method of a RandomAccessFile with r mode can only read data from a file into a buffer. An attempt to write data into this channel will throw java.nio.channels.NonWritableChannelException.

RandomAccessFile raf = new RandomAccessFile("D:\\test1.txt", "r");
FileChannel channel = raf.getChannel();
channel.read(ByteBuffer.allocate(10)); //OK
channel.write(ByteBuffer.allocate(10)); //java.nio.channels.NonWritableChannelException

Create a FileChannel instance by using Paths

We could open a file channel based on java.nio.Path. There is a list OpenOption that we could choose to determine the accessibility of this file channel instance. If neither APPEND or WRITE option is specified for the file channel then an attempt to write data into this channel will throw java.nio.channels.NonWritableChannelException.

Path path = Paths.get("D:\\test.txt");
EnumSet<StandardOpenOption> options
   = EnumSet.of(StandardOpenOption.CREATE_NEW, StandardOpenOption.READ);
FileChannel channel = FileChannel.open(path, options);
channel.read(ByteBuffer.allocate(10)); //OK
channel.write(ByteBuffer.allocate(10)); //java.nio.channels.NonWritableChannelException

What can be done by FileChannel

FileChannel is far more powerful than transitional I/O stream. It implemented a number of interfaces which greatly increase its capabilities rather than just able to read and write file data.

FileChannel is a SeekableByteChannel

Querying file size

Given the file D:\test.txt with content below:

abcdefg
We could query the file size with the codes below is executed,

RandomAccessFile file = new RandomAccessFile("D:\test.txt", "rw");
SeekableByteChannel fileChannel = file.getChannel();

long size = fileChannel.size(); // size in long data type.
System.out.println("Query file size: " + size); // Query file size: 7

The size above is commonly used as the size of the ByteBuffer. This is just working fine if the file size is small.

    ByteBuffer buffer = ByteBuffer.allocate((int) size); // potentially lose precision
    fileChannel.read(buffer);
    printBuffer(buffer);
}

private static void printBuffer(ByteBuffer buffer) {
    buffer.flip();
    while (buffer.hasRemaining()) {
        System.out.print(Character.toChars(buffer.get())[0] + " ");
    }
    System.out.println("");
}

a b c d e f g
In the case where the file size is big, then the intention of just using single ByteBuffer to hold the whole file content will not works. This is because the file size is a long data type, while the byte buffer size is only int data type. To avoid this problem, we could allocate a fixed size byte buffer, and use it repeatably to read the whole file content.

    ByteBuffer buffer = ByteBuffer.allocate(3);
    while (fileChannel.read(buffer) > 0) {
        printBuffer(buffer);
        buffer.clear();
    }

a b c d e f g

Querying and modifying current position

FileChannel also maintains a moving index called current position which indicating the next element in the byte sequence of the file that to be read/wrote. This "current position" could be queried and modified.

    ByteBuffer buffer = ByteBuffer.allocate(3);
    while (fileChannel.read(buffer) > 0) {
        System.out.println("Query current position: " + fileChannel.position());
        printBuffer(buffer);
        buffer.clear();
    }

Query current position: 3
a b c
Query current position: 6
d e f
Query current position: 7
g
The codes below modifies the "current position" to a position which lesser than file size. Therefore, data is read start from that position.

    ByteBuffer buffer = ByteBuffer.allocate(3);
    fileChannel.position(2);
    while (fileChannel.read(buffer) > 0) {
        System.out.println("Query current position: " + fileChannel.position());
        printBuffer(buffer);
        buffer.clear();
    }

Query current position: 5
c d e
Query current position: 7
f g
If we set the "current position" to a position which greater than file size and try to read the file, then nothing is read into buffer and the read() operation returns -1 which indicating the end-of-file.

    ByteBuffer buffer = ByteBuffer.allocate(3);
    fileChannel.position(10); // new position greater than file size.
    System.out.println(fileChannel.read(buffer2)); //reading returns end of file

-1

If we set the "current position" to a position which greater than file size and try to write to the file. The file size will get expanded according to the new content written into the file.

    System.out.println("Current file size: " + fileChannel.size());
    fileChannel.position(10); // new position greater than file size.
    fileChannel.write(ByteBuffer.wrap("klmn".getBytes("UTF-8")));
    System.out.println("New file size: " + fileChannel.size());

Current file size: 7
New file size: 14

Truncating file

We could truncate a file by setting the new size to the FileChannel that connected to the file. Any byte beyond the new size will be removed.

    System.out.println("Current file size: " + fileChannel.size());
    fileChannel.truncate(5);
    System.out.println("New file size: " + fileChannel.size());

    ByteBuffer buffer = ByteBuffer.allocate((int) fileChannel.size());
    fileChannel.read(buffer);
    printBuffer(buffer);

Current file size: 7
New file size: 5
a b c d e
What will happen if the "current position" is greater than the given size? After truncation, the "current position" will be set to the given size, which is the last index in the byte sequence of the file channel.

    System.out.println("Current file size: " + fileChannel.size());
    fileChannel.position(6);

    // position is greater than new size
    System.out.println("Current position: " + fileChannel.position());
    fileChannel.truncate(5);
    System.out.println("New file size: " + fileChannel.size());

    // position is set to the new size
    System.out.println("New position: " + fileChannel.position()); 

Current file size: 7
Current position: 6
New file size: 5
New position: 5
If the given size is equals or greater than current file size, then nothing will be truncated.

    System.out.println("Current file size: " + fileChannel.size());
    fileChannel.truncate(fileChannel.size());
    System.out.println("New file size: " + fileChannel.size());

Current file size: 7
New file size: 7

FileChannel is a GatheringByteChannel and ScatteringByteChannel

FileChannel is able to read/write a sequence of bytes from/to one or more byte buffers in just a single invocation. This is useful when we would like to treats the buffers as different segments of the byte sequence and process them differently right after we perform a read/write operation. Bear in mind that, FileChannel does not know how the bytes sequence should be segmented. It basically just accepts whatever number of buffers we pass to it. It then performs the read/write operation onto those buffers start from the first one. When the first one is totally read out/filled in, then it move to the next one and so on.

In other words, it is our responsibility to determine the data segmentation. We need to know exactly the data format that we are transferring. It normally starts with fixed size buffer then followed by variable-length buffer. One good example of this is to transferring data via the network, where data has to be in the format of header and body, which could be processed differently.

Gathering data segments into channel

    RandomAccessFile file = new RandomAccessFile("D:\test .txt", "rw");
    GatheringByteChannel fileChannel = file.getChannel();

    ByteBuffer header = ByteBuffer.wrap("header".getBytes());
    ByteBuffer body = ByteBuffer.wrap("body".getBytes());
    fileChannel.write(new ByteBuffer[]{header, body});

Scattering data segments from channel

    RandomAccessFile file = new RandomAccessFile("D:\test1.txt", "rw");
    ScatteringByteChannel fileChannel = file.getChannel();

    ByteBuffer header = ByteBuffer.allocate(6);
    ByteBuffer body = ByteBuffer.allocate(10);
    fileChannel.read(new ByteBuffer[]{header, body});

    System.out.println("Print header...");
    printBuffer(header); //header

    System.out.println("nPrint body...");
    printBuffer(body); //body

Transferring data between file channels

FileChannel provides API which allow us to transfer data from one file to another file directly without involve any intermediate buffer.

    RandomAccessFile doc1 = new RandomAccessFile("D:\doc1.txt", "rw"); //abc
    RandomAccessFile doc2 = new RandomAccessFile("D:\doc2.txt", "rw"); //xyz

    FileChannel doc1Channel = doc1.getChannel();
    FileChannel doc2Channel = doc2.getChannel();

    doc1Channel.transferFrom(doc2Channel, 3, 3); //abcxyz
    doc1Channel.transferTo(0, 3, doc2Channel); //xyzabc


References:
http://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileChannel.html

Friday, July 3, 2015

Hierarchical Visitor Pattern with Code Example

What is Hierarchical Visitor Pattern:

"Represent an operation to be performed on the nodes of a hierarchical object structure. Hierarchical Visitor lets one define new operations without changing the classes of the nodes on which it operates. Hierarchical Visitor overcomes the limitations of the traditional Visitor Pattern by allowing a programmer to track traversal depth and short-circuit branch traversal." - http://c2.com/cgi/wiki?HierarchicalVisitorPattern

Class Diagram:



























Explanation:

Basically, Hierarchical Visitor pattern is an extended version of traditional Visitor pattern. Elements appear as node objects in a hierarchical object structure where a CompositeElement could have zero or many other Elements (either CompositeElement or LeafElement). Unlike traditional Visitor pattern, Hierarchical Visitor pattern is able to perform hierarchical navigation and conditional navigation.

Hierarchical Navigation:

The HierarchicalVisitor has 2 visit methods to process each CompositeElement object. In other words, each CompositeElement is visited twice by the same HierarchicalVisitor.
  • visitEnter()HierarchicalVisitor carries out the operation on the CompositeElement once the CompositeElement accepts this HierarchicalVisitor. After that, the HierarchicalVisitor leaves the CompositeElement and visits its child Element(s). The same process happens recursively to all the child Element(s) of each level that under this particular CompositeElement in the hierarchy until the LeafElement.
  • visitExit() - After all child Element(s) of the mentioned CompositeElement are processed, HierarchicalVisitor then comes back to the CompositeElement and carries out the remaining operation on this CompositeElement right before it leave this CompositeElement completely. 
The HierarchicalVisitor has only 1 visit method to process a LeafElement object.
  • visitLeaf()HierarchicalVisitor carries out the operation on the LeafElement once the LeafElement accepts this HierarchicalVisitor. No more navigation to next level because LeafElement has no child Element

Conditional Navigation:

All visit methods return a boolean flag to indicate if the nodes traversal should be continued.
  • visitEnter() - A boolean flag is returned to indicate if traverse to child Element(s) (if any) is required. 
  • visitExit() - A boolean flag is returned to indicate if traverse to next sibling CompositeElement (if any) is required.
  • visitLeaf() - A boolean flag is returned to indicate if traverse to next sibling LeafElement (if any) is required.
This is very useful because full traversal of a hierarchical object structure is not necessary. Whenever a condition is met, HierarchicalVisitor could skip from visiting the child Element(s), or next sibling Element, which lead to optimised navigation.

Participants:

Element (Employee): The common base for CompositeElement and LeafElement. An interface/class that defined the accept() protocol.

import com.hauchee.visitorpattern.visitor.IHierarchicalVisitor;

public abstract class Employee {
    
    private final String name;
    private final int salary;

    public Employee(String name, int salary) {
        this.name = name;
        this.salary = salary;
    }

    public int getSalary() {
        return salary;
    }

    public String getName() {
        return name;
    }
    
    public abstract boolean accept(IHierarchicalVisitor v);
}


CompositeElement (HierarchicalNode): Element that could contain child elements, either CompositeElement or LeafElement. It implements the accept() method so that the visitor could perform an operation on it. It also makes it child elements to accept the visitor, so that they could be visited and processed.

import com.hauchee.visitorpattern.visitor.IHierarchicalVisitor;
import java.util.List;

public class HierarchicalNode extends Employee {

    private List<Employee> subordinates;

    public HierarchicalNode(String name, int score) {
        super(name, score);
    }

    public void setSubordinate(List<Employee> subordinates) {
        this.subordinates = subordinates;
    }

    @Override
    public boolean accept(IHierarchicalVisitor v) {
        if (v.visitEnter(this)) {
            if (subordinates != null) {
                for (Employee cn : subordinates) {
                    boolean collectedAll = cn.accept(v);
                    if (collectedAll) {
                        break;
                    }
                }
            }
        }
        return v.visitExit(this);
    }
}


LeafElement (LeafNode): Element that does not have any child element. It implements the accept() method so that the visitor could perform an operation on it only.

import com.hauchee.visitorpattern.visitor.IHierarchicalVisitor;

public class LeafNode extends Employee {

    public LeafNode(String name, int score) {
        super(name, score);
    }

    @Override
    public boolean accept(IHierarchicalVisitor v) {
        return v.visitLeaf(this);
    }
}


HierarchicalVisitor (IHierarchicalVisitor): An interface that defined the protocol to visit CompositeElement and LeafElement.

import com.hauchee.visitorpattern.element.HierarchicalNode;
import com.hauchee.visitorpattern.element.LeafNode;

public interface IHierarchicalVisitor {

    boolean visitEnter(HierarchicalNode node);

    boolean visitExit(HierarchicalNode node); 

    boolean visitLeaf(LeafNode node);
}


ConcreteVisitor: HierarchicalVisitor implementation that perform operations on CompositeElement and LeafElement.

ReportingLineVisitor implements the operation to be carried out during hierarchical navigation.

import com.hauchee.visitorpattern.element.HierarchicalNode;
import com.hauchee.visitorpattern.element.LeafNode;
import java.util.ArrayList;
import java.util.List;

/**
 * Concrete visitor which collects all the reporting lines.
 */
public class ReportingLineVisitor implements IHierarchicalVisitor {

    private final List<String> reportingLines = new ArrayList<>();
    private final StringBuilder reportingLine = new StringBuilder();
    
    @Override
    public boolean visitEnter(HierarchicalNode node) {
        reportingLine.append(node.getName()).append("/");
        /**
         * Always true to continue traverse to the node's
         * child nodes (if any).
         */
        return true;
    }

    @Override
    public boolean visitExit(HierarchicalNode node) {
        // When exit from the node, remove its name from reportingLine.
        reportingLine.delete(reportingLine.lastIndexOf(node.getName()),
                reportingLine.length() + 1);
        /**
         * Return false to continue traverse to the node's sibling.
         * Only return true when all nodes were processed.
         */
        return reportingLine.length() == 0; 
    }

    @Override
    public boolean visitLeaf(LeafNode node) {
        reportingLines.add(reportingLine.toString() + node.getName());
        /**
         * Return false to continue traverse to the node's sibling.
         */
        return false;
    }

    public List<String> getPaths() {
        return reportingLines;
    }
}

TotalSalaryVisitor implements the operation to be carried out during conditional navigation.

import com.hauchee.visitorpattern.element.Employee;
import com.hauchee.visitorpattern.element.HierarchicalNode;
import com.hauchee.visitorpattern.element.LeafNode;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

/**
 * Concrete visitor which accumulate all salary of a given reporting line.
 */
public class TotalSalaryVisitor implements IHierarchicalVisitor {

    private final List<String> reportingLine;
    private int salary;
    
    public TotalSalaryVisitor(String reportingLine) {
        this.reportingLine = new ArrayList<>(Arrays.asList(reportingLine.split("/")));
    }

    public int getSalary() {
        return salary;
    }
    
    private boolean process(Employee employee) {
        if (reportingLine.isEmpty()) {
            return false;
        }
        if (employee.getName().equals(reportingLine.get(0))
                && reportingLine.size() >= 1) {
            salary += employee.getSalary();
            reportingLine.remove(0);
            return true;
        }
        return false;
    }
    
    @Override
    public boolean visitEnter(HierarchicalNode node) {
        return process(node);
    }  

    @Override
    public boolean visitExit(HierarchicalNode node) {
        return reportingLine.isEmpty();
    }

    @Override
    public boolean visitLeaf(LeafNode node) {
        return process(node);
    }
}


Client (App): The client application that has objects in hierarchical object structure and make use of different ConcreteVisitor(s) to perform operations on those objects.

import com.hauchee.visitorpattern.element.HierarchicalNode;
import com.hauchee.visitorpattern.element.Employee;
import com.hauchee.visitorpattern.element.LeafNode;
import com.hauchee.visitorpattern.visitor.ReportingLineVisitor;
import com.hauchee.visitorpattern.visitor.TotalSalaryVisitor;
import java.util.ArrayList;
import java.util.List;

public class App {

    public static void main(String[] args) {

        /**
         * Construct employees reporting line in hierarchical structure as
         * below.
         *
         * Boss 
         *  - HR Manager 
         *      - HR Staff 1 
         *      - HR Staff 2 
         *      - HR Staff 3 
         *  - Sales Manager 
         *      - Sales Staff 1 
         *      - Sales Staff 2
         *  - Finance Manager
         */
        HierarchicalNode boss = new HierarchicalNode("Boss", 1000000);

        List<Employee> management = new ArrayList<>();

        HierarchicalNode hrManager = new HierarchicalNode("HR Manager", 6000);
        management.add(hrManager);
        List<Employee> hrStaffs = new ArrayList<>();
        hrStaffs.add(new LeafNode("HR Staff 1", 2000));
        hrStaffs.add(new LeafNode("HR Staff 2", 2500));
        hrStaffs.add(new LeafNode("HR Staff 3", 2500));
        hrManager.setSubordinate(hrStaffs);

        HierarchicalNode salesManager = new HierarchicalNode("Sales Manager", 8000);
        management.add(salesManager);
        List<Employee> salesStaffs = new ArrayList<>();
        salesStaffs.add(new LeafNode("Sales Staff 1", 4000));
        salesStaffs.add(new LeafNode("Sales Staff 2", 5500));
        salesManager.setSubordinate(salesStaffs);

        management.add(new LeafNode("Finance Manager", 9000));

        boss.setSubordinate(management);

        /**
         * Print all reporting lines.
         */
        ReportingLineVisitor rlVisitor = new ReportingLineVisitor();
        boss.accept(rlVisitor);
        for(String path : rlVisitor.getPaths()) {
            System.out.println(path);
        }
        
        /**
         * Total up the salary of employees in a specific reporting line.
         */
        TotalSalaryVisitor tsVisitor = new TotalSalaryVisitor("Boss/Sales Manager");
        boolean success = boss.accept(tsVisitor);
        if (success) {
            System.out.println(tsVisitor.getSalary());
        }
    }
}

Boss/HR Manager/HR Staff 1
Boss/HR Manager/HR Staff 2
Boss/HR Manager/HR Staff 3
Boss/Sales Manager/Sales Staff 1
Boss/Sales Manager/Sales Staff 2
Boss/Finance Manager
1008000


References:
http://c2.com/cgi/wiki?HierarchicalVisitorPattern
https://en.wikipedia.org/wiki/Visitor_pattern