space alocation for matrices

hi

i run tomlab under matlab.

i need to build several matrices in my code. when i pre-allocate memory with zeros(...) i come up with the so famous memory problem.
i can also allocate memory with spalloc or sparse. however, they really prolongs the time to an untolarable degree that the code finds a solution to the problem .

can you suggest a remedy for this difficulty? Since solution time is also important i need to find a way to pre-allocate space for my matrices that will not cause neither a memory problem or a time increase.

thanks.

below is part of my code to give some insight to the possible helpers.

% grad1 = zeros(n,N);
% grad2 = zeros(n,N);
% grad3 = zeros(n,N);
% grad4 = zeros(n,N);
% grad5 = zeros(n,N);
% grad6 = zeros(n,N);
% grad7 = zeros(n,N);
% grad8 = zeros(n,N);
% grad9 = zeros(n,N);
% grad10= zeros(n,N);

% use to allocate space for matrices when N is big
grad1 = spalloc(n,N,N*13);
grad2 = spalloc(n,N,N*13);
grad3 = spalloc(n,N,N*13);
grad4 = spalloc(n,N,N*13);
grad5 = spalloc(n,N,N*13);
grad6 = spalloc(n,N,N*13);
grad7 = spalloc(n,N,N*13);
grad8 = spalloc(n,N,N*13);
grad9 = spalloc(n,N,N*13);
grad10 = spalloc(n,N,N*13);

mtx = zeros(row,1);
for i=1:N
mtx(i) = -1; %derivative of zj
grad1(:,i) = [0;
-0.3717*(sample(i,4)+x(4)) - 0.00931*x(10);
-0.484*x(9);
-0.3717*(sample(i,2)+ x(2));
0;
0.01343*x(10);
0;
0;
-0.484*(sample(i,3)+x(3));
-0.00931*(sample(i,2)+x(2))+ 0.01343*(sample(i,6)+x(6));
0;
-1
mtx];
mtx = zeros(row,1);
end

for i=1:N
mtx(i) = -1;
grad2(:,i) = [-0.0159*(sample(i,2)+x(2))-0.188*x(8);
-0.0159*(sample(i,1)+x(1))-0.019*(sample(i,7)+x(7));
.
.
.
.

thx all Start a New Thread

Hi,

First thing that struck me was maybe treat this as a source coding problem
i.e. exploit the characteristics of your data.
If you know the occurrences of the zeros in your sparse matrix, then maybe
consider storing on those rows(vectors). The for loops seem to suggest that
grad1 has always zeros in the first row. Why not exploit this ?

Even better option would be to implement this in IT++
http://itpp.sourceforge.net/current/index.html

Other options:
Maybe split your code to consider using one sparse matrix at a time and do
some house keeping by clearing up memory. Maybe use load/save as mat files.
If your loading gets slow use -v6 option.
Or export your code to run as a executable using MATLAB's deploytool. Have a
look at the documentation to see if this suits your needs.

Good luck.
Cheers,
Tilak

On Fri, Dec 18, 2009 at 10:24 AM, wrote:

> hi
>
> i run tomlab under matlab.
>
> i need to build several matrices in my code. when i pre-allocate memory
> with zeros(...) i come up with the so famous memory problem.
> i can also allocate memory with spalloc or sparse. however, they really
> prolongs the time to an untolarable degree that the code finds a solution to
> the problem .
>
> can you suggest a remedy for this difficulty? Since solution time is also
> important i need to find a way to pre-allocate space for my matrices that
> will not cause neither a memory problem or a time increase.
>
> thanks.
>
> below is part of my code to give some insight to the possible helpers.
>
> % grad1 = zeros(n,N);
> % grad2 = zeros(n,N);
> % grad3 = zeros(n,N);
> % grad4 = zeros(n,N);
> % grad5 = zeros(n,N);
> % grad6 = zeros(n,N);
> % grad7 = zeros(n,N);
> % grad8 = zeros(n,N);
> % grad9 = zeros(n,N);
> % grad10= zeros(n,N);
>
> % use to allocate space for matrices when N is big
> grad1 = spalloc(n,N,N*13);
> grad2 = spalloc(n,N,N*13);
> grad3 = spalloc(n,N,N*13);
> grad4 = spalloc(n,N,N*13);
> grad5 = spalloc(n,N,N*13);
> grad6 = spalloc(n,N,N*13);
> grad7 = spalloc(n,N,N*13);
> grad8 = spalloc(n,N,N*13);
> grad9 = spalloc(n,N,N*13);
> grad10 = spalloc(n,N,N*13);
>
> mtx = zeros(row,1);
> for i=1:N
> mtx(i) = -1; %derivative of zj
> grad1(:,i) = [0;
> -0.3717*(sample(i,4)+x(4)) - 0.00931*x(10);
> -0.484*x(9);
> -0.3717*(sample(i,2)+ x(2));
> 0;
> 0.01343*x(10);
> 0;
> 0;
> -0.484*(sample(i,3)+x(3));
> -0.00931*(sample(i,2)+x(2))+ 0.01343*(sample(i,6)+x(6));
> 0;
> -1
> mtx];
> mtx = zeros(row,1);
> end
>
> for i=1:N
> mtx(i) = -1;
> grad2(:,i) = [-0.0159*(sample(i,2)+x(2))-0.188*x(8);
> -0.0159*(sample(i,1)+x(1))-0.019*(sample(i,7)+x(7));
> .
> .
> .
> .
>
> thx all
>
> Start a New Thread