Author Topic: best fit ......poisson and students t distribution  (Read 313 times)

mikeburr

  • Regular Member
  • *
  • Posts: 29
best fit ......poisson and students t distribution
« on: October 17, 2017, 09:37:28 PM »
has any one got a poisson and/or students T distribution i can borrow [ i need the source unfortunately as im going to have to hack it probably to do a least square approx ]
Heres why :-
im trying to find a best fit for a skewed set of data  .. as the data set gets larger i suppose i could use a normal distribution but being limited to 32 bit
im already hitting the 2 gig limit of the O/S   so id rather use some thing better tuned to the data
Its true i am being a bit lazy by asking but i very rarely use the floating point stuff
and it would therefore take a while to do
Being a tad old now i dont want to die trying to get one running ...
regards mike burr 


hutch--

  • Administrator
  • Member
  • ******
  • Posts: 4883
  • Mnemonic Driven API Grinder
    • The MASM32 SDK
Re: best fit ......poisson and students t distribution
« Reply #1 on: October 17, 2017, 10:10:51 PM »
The lab is for code, not requests.
hutch at movsd dot com
http://www.masm32.com    :biggrin:  :biggrin:

jj2007

  • Member
  • *****
  • Posts: 7662
  • Assembler is fun ;-)
    • MasmBasic
Re: best fit ......poisson and students t distribution
« Reply #2 on: October 17, 2017, 10:23:02 PM »
Google for gsl_ran_poisson, maybe you can find a C example.

mikeburr

  • Regular Member
  • *
  • Posts: 29
Re: best fit ......poisson and students t distribution
« Reply #3 on: October 17, 2017, 10:33:10 PM »
@@hutch
sorry
@@ jj
thanks   bag of worms !!! problems with the GNU gamma function i believe ... oh well ill have to write it myself i guess
regards mike b
 

HSE

  • Member
  • ****
  • Posts: 545
  • <AMD>< 7-32>
Re: best fit ......poisson and students t distribution
« Reply #4 on: October 17, 2017, 10:44:01 PM »
I Mike!
You can search in the forum "SmplMath". With that macros system you can write equations easily.
Regards. HSE.

mikeburr

  • Regular Member
  • *
  • Posts: 29
Re: best fit ......poisson and students t distribution
« Reply #5 on: October 18, 2017, 12:39:31 AM »
Dear HSE
i have to say im not that confident at handling all the marvelous stuff Raymond has provided. The computation of probability is relatively simple  but the fitting it to the data is not
Example data set
[3,2];[2,5];[4,6,1];[2,11,2];[4,13,5];[3,17,10]'[4,22,15,1];[2,27,25,2]  ........[2,435,10759,86440,284674,411687,258279,64131,5033,65],[4,544,11593,92883,316709,476204,312411,82577,7121,110],....
where the length of sequence extends according to a rule ive established as  1+ [n(n+1)/2]  The data you'll notice has a very Poisson like distribution
BUT the  k and lambda are not discrete so in order to match the distribution to the data it will be necessary to vary k and lambda to match the profile
I suppose people might have wondered reading the first post what i was doing as Poisson is not immediately obvious as a best fit solution !!!
i hope this sort of  explains what im thinking of doing in trying to establish a connection between non linear and discrete functions this way
regards mike b
 

jj2007

  • Member
  • *****
  • Posts: 7662
  • Assembler is fun ;-)
    • MasmBasic
Re: best fit ......poisson and students t distribution
« Reply #6 on: October 18, 2017, 01:36:50 AM »
has any one got a poisson and/or students T distribution i can borrow

Do you need data or what? Getting Poisson data is not a big deal with GSL:

include \masm32\MasmBasic\MasmBasic.inc         ; download
; libgsl.dll download : open Binaries zip, and extract the two files
; libgsl.dll and libgslcblas.dll to \masm32\MasmBasic\GnuScLib\DLL
; define your required gsl function(s) using the syntax of the GNU Scientific Library Reference
  gslvar int gsl_rng_mt19937()
  gslvar int gsl_rng_taus()
  gsl    int gsl_rng_alloc(int data)
  gsl    int gsl_ran_poisson (const gsl_rng * r, double mu)
  SetGlobals rng, mu:REAL8=10.0
  Init
  mov rng, gsl_rng_alloc(gsl_rng_mt19937())
  Print "mt19937:", Tb$
  For_ ct=0 To 19
        Print Str$("%i ", gsl_ran_poisson(rng, mu))
  Next
  mov rng, gsl_rng_alloc(gsl_rng_taus())
  Print CrLf$, "taus:   ", Tb$
  For_ ct=0 To 19
        Print Str$("%i ", gsl_ran_poisson(rng, mu))
  Next
  Inkey CrLf$, "--- hit any key ---"
EndOfCode


Output:
Code: [Select]
mt19937:        15 6 9 9 5 8 11 9 11 5 10 8 15 8 10 14 4 3 14 10
taus:           7 6 12 8 7 14 8 16 6 8 8 10 10 10 15 7 12 10 10 13

Otherwise, if you have the data already, have a look at Least-Squares Fitting and tell us what you need.

mikeburr

  • Regular Member
  • *
  • Posts: 29
Re: best fit ......poisson and students t distribution
« Reply #7 on: October 18, 2017, 04:12:21 AM »
@@jj   
thanks very much jj i will need data for testing the distribution is correct so thats brilliant   thank you  Ill be running the distribution using a least squares system to try and align it to the data i put on the post [ which is both invariant and i think is independant .. if not then we've all got to rethink number theory !!!!!] 
@@hse
what a brilliant facility QWORD has provided on source forge ... [wasnt sure whether i should put the link up or not as it took a quick search to find everything ]
though im glad i only recently joined as i might have used the very comprehensive linked list whereas i developed a file system pertinent to each of the programs ive developed  and this has not ony furthered my understanding of the computer os and hardware but acts as a foundation for further development [trade off between a better easier system to use and something you understand because you had to make it ]
regards    mike b

HSE

  • Member
  • ****
  • Posts: 545
  • <AMD>< 7-32>
Re: best fit ......poisson and students t distribution
« Reply #8 on: October 18, 2017, 12:19:16 PM »
Just remember, with linealization you obtain least mean square error of transformed data. Better adjustment of true data usually requiere no lineal methods like Levenberg-Marquardt.

jj2007

  • Member
  • *****
  • Posts: 7662
  • Assembler is fun ;-)
    • MasmBasic
Re: best fit ......poisson and students t distribution
« Reply #9 on: October 18, 2017, 05:54:27 PM »
non-linear methods like Levenberg-Marquardt.

https://www.gnu.org/software/gsl/manual/html_node/Example-programs-for-Nonlinear-Least_002dSquares-Fitting.html
Quote
The main part of the program sets up a Levenberg-Marquardt solver and some simulated random data. The data uses the known parameters (5.0,0.1,1.0) combined with Gaussian noise (standard deviation = 0.1) over a range of 40 timesteps. The initial guess for the parameters is chosen as (0.0, 1.0, 0.0).

HSE

  • Member
  • ****
  • Posts: 545
  • <AMD>< 7-32>
Re: best fit ......poisson and students t distribution
« Reply #10 on: October 18, 2017, 11:18:13 PM »
Perfect JJ  :t

Most of the time, I use my own Least Mean Square method because is very easy to make it in assembly. For Levenberg-Marquardt I use R-project wich is a good scientific reference (also free). (I don't know if Gunther worked in that project). 
« Last Edit: October 19, 2017, 04:21:33 AM by HSE »

mikeburr

  • Regular Member
  • *
  • Posts: 29
Re: best fit ......poisson and students t distribution
« Reply #11 on: November 08, 2017, 06:29:32 AM »
thanks very much for your kind assistance and recommendations ..  i should have looked at the things id done here before as it was pretty much glaringly obvious .. anyway the answer is a gamma like distribution of the form [ (t!)^2 (t+1)] / [((t-k)!)^2 * (k!)*2 * (k+1)]  which prob sounds a bit elaborate but wasnt too hard to find .. I think i now have the solution to a thing called Ramseys Theorem .Paul Erdos made an exagerated quote as to the difficulty of computing R(5) and thence  R(6) which is quite amusing
regards   mike b

mikeburr

  • Regular Member
  • *
  • Posts: 29
Re: best fit ......poisson and students t distribution
« Reply #12 on: November 08, 2017, 07:40:16 AM »
this places R(5) at 43.532 coincidentally the same serial number as the Lincoln Premier Dinky toy

mikeburr

  • Regular Member
  • *
  • Posts: 29
Re: best fit ......poisson and students t distribution
« Reply #13 on: November 08, 2017, 09:31:00 AM »
the others for what its worth are R(6) = 137 ,  R(7) = 363 or maybe 364 , R(8) = 922,  R(9) = 2352, r(10) = 5750
regards mike b