This Question is Answered

1 "helpful" answer available (2 pts)
13 Replies Last post: Jan 15, 2008 11:23 AM by att  
att   22 posts since
Oct 29, 2007
Currently Being Moderated

Dec 18, 2007 6:07 PM

Writing performance

Hello again.

I have some questions about writing performance.

 

I try to populate a area in my attached excel-file wb.xls. Seen marked with yellow background in the document.

Im getting an average writing speed of about 204 cells per second, with my code.

To read the 23*23 area it takes about 2,5 seconds, its slow.

And when repeating this process (as part of a simulation) it take a whole lot of time.

 

What in the fillWithArray()-function takes time?

How much overhead is there in performing the function fillWithArray()?

 

Could I call native functions / use native peers to make it faster?

 

The million dollar question is: How can I make it to get faster?

 

Here's the code for the operation:


public class WriteReadTest {
    private Workbook workbook;
    private Application application;
    public WriteReadTest() {
        try{
            long time1 = System.currentTimeMillis();
            long time2 = System.currentTimeMillis();
            DecimalFormat df = new DecimalFormat("0.##");
            Random rand = new java.util.Random();
            ArrayList<Object> dl = new ArrayList<Object>();
            
            time1 = System.currentTimeMillis();
            application = new Application();///Makes a new application
            File xlsFile = new File("wb.xls");
            workbook = application.openWorkbook(xlsFile);
            time2 = System.currentTimeMillis();
            long sum= (time2-time1);
            System.out.println("Started application and opened the wb.xls in " + sum + " ms");
            
            Cell startCell = workbook.getWorksheet(1).getCell(46,2); //startCell in Sheet1
            
            double[][] data = new double[23][23];//A twodim table with random values
            double dimension = 23*23; //dimension of table
            data = new double[23][23];
            for (int i=0;i<23;i++){
                for (int j=0;j<23;j++){
                    data+[j] = rand.nextInt(10);
                }
            }
            
            Worksheet w = workbook.getWorksheet(1);
            
            time1 = System.currentTimeMillis();
            w.fillWithArray(startCell, data); //#### Fills with array. 
            time2 = System.currentTimeMillis();
            
            sum= (time2-time1);
            double sek = (sum/1000.00);
            System.out.println(sek);
            
            System.out.println("Writing filling range 'B46:X68' took " + sum + " ms. Writingspeed per cell : " + df.format(dimension/(sum/1000.00)) + " cells per second");

            workbook.saveCopyAs(new File("data/wb_runned.xls"));//Optionally, to see what numbers got inserted
            workbook.close(false);
            application.close();
        } catch (FileNotFoundException ex) {
            ex.printStackTrace();
            workbook.close(false);
            application.close();
        } catch (ExcelException ex) {
            ex.printStackTrace();
            workbook.close(false);
            application.close();
        }catch (IOException ex) {
            ex.printStackTrace();
        }
    }
    
    public static void main(String[] args) {
        WriteReadTest wrt = new WriteReadTest();

    }
            

 

Message was edited by: att

Attachments:
Mike McMaster   61 posts since
Aug 24, 2007
Currently Being Moderated
1. Dec 20, 2007 4:03 AM in response to: att
Re: Writing performance

I'm interested in this subject as well. The write performance is reasonable for our needs, but any improvements would be really useful.

Igor Novikov TeamDev Ltd. 225 posts since
Apr 24, 2006
Currently Being Moderated
4. Dec 21, 2007 1:01 PM in response to: att
Re: Writing performance

Hi,

 

Of course, you can try using VBA scripting to resolve the issue. Small example which demonstrates VBA script with input parameters you can find on following link:

 

http://support.teamdev.com/thread.jspa?threadID=1857&tstart=0

 

Also you can generate VBA script at runtime which already contains array data.

 

Another possible solution is a stubs usage. Range native peer has method setValue2(Variant param). It's an equivalent of VBA Range("...").Value

For example:

 


workbook.getWorksheet(1).getRange("A1:IV100").getPeer().setValue2(new Variant(3.14159));
workbook.getWorksheet(1).getRange("C3").getPeer().setValue2(new Variant("Tester!"));

 

This pseudocode is Java equivalent of your VBA Sample_Func(). All the approaches are faster than simple iteration over Cell objects because they use bulk operations.

 

I hope this helps.

 

Sincerely,

 

Igor Novikov

Mike McMaster   61 posts since
Aug 24, 2007
Currently Being Moderated
6. Jan 8, 2008 3:18 AM in response to: att
Re: Writing performance

Here is my attempt at writing a 2D array. Please see the attached file JExcelTest.

 

However, when I run this code, I get this exception:

 

Exception in thread "main" com.jniwrapper.win32.com.ComException: COM object method returns error code: 0x8007000E; E_OUTOFMEMORY (Not enough storage is available to complete this operation.)

at com.jniwrapper.win32.automation.types.SafeArray.create(SourceFile:432)

at com.jniwrapper.win32.automation.types.SafeArray.&lt;init&gt;(SourceFile:144)

at com.abwg.client.jexcel.JExcelTest.main(JExcelTest.java:31)

 

 

 

 

 

 

 

Is this normal? Seems strange that it would run out of memory just by initializing the SafeArray. Are there any ways to improve this?

 

Thanks,

 

Mike

Attachments:
Mike McMaster   61 posts since
Aug 24, 2007
Currently Being Moderated
7. Jan 9, 2008 3:48 AM in response to: Mike McMaster
Re: Writing performance

Looks like I figured it out. See attached JExcelTest.java.

 

For some reason, SafeArray throws the E_OUTOFMEMORY error when using Str, but not when using BStr. Also, LongInt and DoubleFloat work as well. I use DoubleFloat in the example.

 

Hope this helps, att. I know it will help me.

 

Thanks,

Mike

Attachments:
Mike McMaster   61 posts since
Aug 24, 2007
Currently Being Moderated
8. Jan 10, 2008 1:09 AM in response to: Mike McMaster
Re: Writing performance

A SafeArray of Variants also works. I found this to be the most flexible as Variant supports many types -- you can pass an int or a String, for example, and Excel will interpret it correctly.

 

Mike

Igor Novikov TeamDev Ltd. 225 posts since
Apr 24, 2006
Currently Being Moderated
9. Jan 10, 2008 5:48 PM in response to: Mike McMaster
Re: Writing performance

 

Hi Mike,

 

 

Thank you  for the samples. It seems we need extending JExcel API for such bulk  operations. It should simplify similar tasks.

 

 

Sincerely,

 

 

Igor Novikov

 

 

Mike McMaster   61 posts since
Aug 24, 2007
Currently Being Moderated
11. Jan 14, 2008 7:36 PM in response to: att
Re: Writing performance

Hi att,

 

Yeah, the set functionality is a little nonstandard (for Java at least, not sure about other languages). When you call the SafeArray constructor and SafeArray.set(), the idea is that you pass an integer array whose dimensions match the array you're writing to, and whose values match the address you want to write to.

 

Here's one way to create and populate a 2D array:

 

final int ARRAY_SIZE = 100;

int count = 0;

final SafeArray array = new SafeArray(new int[] {ARRAY_SIZE, ARRAY_SIZE}, Variant.class);

int[] rowCol = new int[2];

for (int i = 0; i < ARRAY_SIZE; i++) {

  for (int j = 0; j < ARRAY_SIZE ; j++) {

    Variant val = new Variant(count++);

    rowCol[0] = i;

    rowCol[1] = j;

    array.set(rowCol, val);

  }

}

 

For a 1D array, it looks like you can just pass the index:

 

final SafeArray array = new SafeArray(invArray.length, Variant.class);

for (int index = 0; i < invArray.length; i++) {

  array.set(index, new Variant(invArray[index]));

}

Mike McMaster   61 posts since
Aug 24, 2007
Currently Being Moderated
12. Jan 14, 2008 7:38 PM in response to: att
Re: Writing performance

I think the new forum software needs a CODE tag. :-P

More Like This

  • Retrieving data ...