Logs: liberachat/#haskell
| 2026-03-07 10:53:30 | <Guest89> | I just don't see how garbage collection can dominate the runtime so much |
| 2026-03-07 10:55:14 | <haskellbridge> | <sm> well, how much memory does +RTS -s say is being allocated ? |
| 2026-03-07 10:55:26 | <Guest89> | is it possible to paste pictures over IRC? |
| 2026-03-07 10:55:31 | <Guest89> | or should I just put it in writing |
| 2026-03-07 10:55:46 | <haskellbridge> | <sm> it's a lot easier if you use the matrix room |
| 2026-03-07 10:55:46 | × | arandombit quits (~arandombi@user/arandombit) (Ping timeout: 248 seconds) |
| 2026-03-07 10:55:58 | <Guest89> | not familiar |
| 2026-03-07 10:56:37 | <Guest89> | I guess I'll just write it down: |
| 2026-03-07 10:56:45 | <Leary> | Guest89: The problem is less likely to be allocations than unnecessary retention or unwanted thunks bloating your representation. Allocating is almost free, holding onto it is what costs you. In any case, I would start by heap profiling by type, which doesn't actually require a profiling build. |
| 2026-03-07 10:57:14 | <Leary> | @where paste |
| 2026-03-07 10:57:14 | <lambdabot> | Help us help you: please paste full code, input and/or output at e.g. https://paste.tomsmeding.com |
| 2026-03-07 10:57:29 | → | Beowulf joins (florian@sleipnir.bandrate.org) |
| 2026-03-07 10:57:34 | <Guest89> | https://paste.tomsmeding.com/xZZPhSCR |
| 2026-03-07 10:58:11 | <Guest89> | the only thing I haven't tried has to force computations in different places |
| 2026-03-07 10:58:49 | × | ChaiTRex quits (~ChaiTRex@user/chaitrex) (Ping timeout: 258 seconds) |
| 2026-03-07 10:58:57 | → | CiaoSen joins (~Jura@2a02:8071:64e1:da0:5a47:caff:fe78:33db) |
| 2026-03-07 10:59:04 | <Guest89> | my reference implementation generates only a few megabytes of data by comparison but again it's not comparable 1:1 |
| 2026-03-07 10:59:22 | × | hiecaq quits (~hiecaq@user/hiecaq) (Quit: ERC 5.6.0.30.1 (IRC client for GNU Emacs 30.2)) |
| 2026-03-07 10:59:28 | <haskellbridge> | <sm> how do you do that Leary ? |
| 2026-03-07 10:59:50 | <Guest89> | it's one of the -l(x) RTS settings |
| 2026-03-07 11:00:49 | <Leary> | -hT: https://downloads.haskell.org/ghc/latest/docs/users_guide/profiling.html#rts-options-heap-prof |
| 2026-03-07 11:00:51 | <Guest89> | sorry, -h |
| 2026-03-07 11:01:03 | <Guest89> | i'll give it a whirl |
| 2026-03-07 11:01:49 | <Guest89> | I have some plots from using -hc that tells me which functions allocate the most but to be honest they're not particularly surprising in that department |
| 2026-03-07 11:02:32 | → | ChaiTRex joins (~ChaiTRex@user/chaitrex) |
| 2026-03-07 11:02:52 | <Guest89> | also I've been relying on using eventlog2html but it seems to break fairly easily. are there any other options for visualizing the profiles? |
| 2026-03-07 11:08:11 | → | merijn joins (~merijn@host-cl.cgnat-g.v4.dfn.nl) |
| 2026-03-07 11:11:08 | <Guest89> | seems the biggest allocations come from primitive types, tuples etc |
| 2026-03-07 11:15:03 | × | merijn quits (~merijn@host-cl.cgnat-g.v4.dfn.nl) (Ping timeout: 265 seconds) |
| 2026-03-07 11:16:09 | <probie> | Beyond the special syntax, tuples aren't really much different from `data T2 a b = T2 a b`, `data T3 a b c = T3 a b c`, `data T4 a b c d = T4 a b c d` etc. |
| 2026-03-07 11:17:07 | <Guest89> | I thought ghc treated them differently? |
| 2026-03-07 11:17:16 | <probie> | Why? |
| 2026-03-07 11:17:43 | <Guest89> | I just understood ghc optimized for tuples in particular over ordinary data constructors |
| 2026-03-07 11:21:47 | <Leary> | Guest89: It might here and there, but that doesn't mean you're better off using them. In particular, they're lazy, so they can easily accumulate big thunks. I suggest replacing such parts of your representation with suitably strict bespoke data declarations. |
| 2026-03-07 11:22:18 | → | merijn joins (~merijn@host-cl.cgnat-g.v4.dfn.nl) |
| 2026-03-07 11:22:46 | <probie> | GHC is capable of doing something like `f x = (x, x+1)`, `g x = let (a, b) = f x in a + b` without actually allocating a tuple (assuming it can inline `f`), but that's the same for any user defined type as well |
| 2026-03-07 11:23:55 | <probie> | If you need multiple return values and can't afford an allocation, look at unboxed tuples (although this is likely overkill) |
| 2026-03-07 11:26:10 | × | wootehfoot quits (~wootehfoo@user/wootehfoot) (Read error: Connection reset by peer) |
| 2026-03-07 11:27:58 | × | merijn quits (~merijn@host-cl.cgnat-g.v4.dfn.nl) (Ping timeout: 276 seconds) |
| 2026-03-07 11:29:16 | <Guest89> | I tried playing around with unboxed tuples and different pragmas like unpacking but they seemed to have varying/counterintuitive results |
| 2026-03-07 11:29:47 | × | Sgeo quits (~Sgeo@user/sgeo) (Read error: Connection reset by peer) |
| 2026-03-07 11:29:55 | <Guest89> | I should probably experiment with it more but it seemed like allocations went down only a little (or at least less than I expected) while somehow runtimes increased slightly |
| 2026-03-07 11:30:20 | <Guest89> | but it kind of feels like I am at the kitchen sink stage in general if I'm being honest |
| 2026-03-07 11:31:40 | <Guest89> | actually, one thing I have been doing in general is guard patterns for unpacking. are they lazy as well or strict the same way pattern matching is? |
| 2026-03-07 11:33:56 | <haskellbridge> | <sm> Guest89 you won't be able to fix it by kitchen sink experimenting, you'll need to dig in and understand. There's likely many causes of space leak |
| 2026-03-07 11:34:24 | <haskellbridge> | <sm> reducing to a simple program you can share, may help |
| 2026-03-07 11:35:18 | <haskellbridge> | <sm> or, commenting out large chunks of your program to see what makes a difference |
| 2026-03-07 11:35:51 | <haskellbridge> | <sm> if you make a profile, people here will help you read it |
| 2026-03-07 11:38:05 | → | merijn joins (~merijn@host-cl.cgnat-g.v4.dfn.nl) |
| 2026-03-07 11:38:34 | <Guest89> | when you say *make* a profile, what do you mean exactly? |
| 2026-03-07 11:38:54 | <haskellbridge> | <sm> I looked it up: stack install --profile PROG; PROG +RTS -P -RTS ... this will save PROG.prof |
| 2026-03-07 11:38:58 | <Guest89> | I've got the files from profiling and some html pages rendered from them if that's what you mean |
| 2026-03-07 11:39:54 | <Guest89> | do you just want a file dump? |
| 2026-03-07 11:40:02 | <haskellbridge> | <sm> do you have a .prof file ? |
| 2026-03-07 11:40:07 | <Guest89> | yes |
| 2026-03-07 11:40:18 | <haskellbridge> | <sm> by all means show it :) |
| 2026-03-07 11:40:51 | <Guest89> | https://paste.tomsmeding.com/AUb2v7Sh |
| 2026-03-07 11:41:02 | → | Square3 joins (~Square@user/square) |
| 2026-03-07 11:41:39 | <haskellbridge> | <sm> lovely. And it might be interesting to run profiterole on that too. |
| 2026-03-07 11:42:17 | <Guest89> | will try |
| 2026-03-07 11:43:01 | <haskellbridge> | <sm> see the entries column.. you have something being called 26 million times, eg. Is that what you'd expect ? Is the data that large ? |
| 2026-03-07 11:43:07 | × | merijn quits (~merijn@host-cl.cgnat-g.v4.dfn.nl) (Ping timeout: 268 seconds) |
| 2026-03-07 11:43:12 | → | arandombit joins (~arandombi@user/arandombit) |
| 2026-03-07 11:43:34 | <Guest89> | so currently on a benchmark that I have (encoding `n-queens`) the number of nodes in my data structure is expected to quadruple for each n but currently space and time seems to increase 10-fold instead |
| 2026-03-07 11:43:46 | <Guest89> | but on that particular run; no, that is excessive |
| 2026-03-07 11:44:02 | <haskellbridge> | <sm> it sounds like something is doing too much work |
| 2026-03-07 11:44:30 | <haskellbridge> | <sm> $wbddApply'' is called half a million times |
| 2026-03-07 11:44:50 | <Guest89> | let me try something for a quick sanity check |
| 2026-03-07 11:44:57 | <haskellbridge> | <sm> $sinsert_$sgo4 14 million. maybe one of those... |
| 2026-03-07 11:45:47 | <haskellbridge> | <sm> (I don't know why these names are obfuscated) |
| 2026-03-07 11:46:07 | <Guest89> | I don't know what they are either |
| 2026-03-07 11:46:16 | <Guest89> | I've only seem them now that I'm running with -P instead of -p |
| 2026-03-07 11:47:02 | <Guest89> | well insert is probably from the data structure I use to maintain a priority queue |
| 2026-03-07 11:48:17 | × | arandombit quits (~arandombi@user/arandombit) (Ping timeout: 268 seconds) |
| 2026-03-07 11:50:02 | <Guest89> | so in this particular example the data is generated from a fold that iteratively uses bddApply on new BDDs, but only 7 times total. the largest BDDs being applied have a few thousand nodes each, which means that the upper bound for bddApply will necessarily be in the millions |
| 2026-03-07 11:50:19 | <Guest89> | but most likely it should be less than that because of short circuiting on a lot of the node combinations |
| 2026-03-07 11:50:44 | <Guest89> | it seems like a lot but I can't dismiss it as unexpected |
| 2026-03-07 11:52:51 | <haskellbridge> | <sm> it looks like there's a lot comparing needed to do an insert. Is it your own custom priority queue ? |
| 2026-03-07 11:54:13 | → | __monty__ joins (~toonn@user/toonn) |
| 2026-03-07 11:54:15 | → | merijn joins (~merijn@62.45.136.136) |
| 2026-03-07 11:56:58 | × | Guest89 quits (~Guest89@185.45.21.144) (Ping timeout: 240 seconds) |
| 2026-03-07 11:58:49 | × | merijn quits (~merijn@62.45.136.136) (Ping timeout: 244 seconds) |
| 2026-03-07 11:58:53 | <haskellbridge> | <sm> got to go.. good luck |
| 2026-03-07 11:59:28 | → | bggd__ joins (~bgg@2a01:e0a:fd5:f510:537e:c033:7f9f:3728) |
| 2026-03-07 12:03:00 | × | fun-safe-math quits (~fun-safe-@97.115.234.213) () |
| 2026-03-07 12:05:04 | → | fun-safe-math joins (~fun-safe-@97.115.234.213) |
| 2026-03-07 12:06:12 | × | Square3 quits (~Square@user/square) (Remote host closed the connection) |
| 2026-03-07 12:06:30 | → | Square joins (~Square@user/square) |
| 2026-03-07 12:09:37 | → | merijn joins (~merijn@host-cl.cgnat-g.v4.dfn.nl) |
| 2026-03-07 12:11:01 | × | gmg quits (~user@user/gehmehgeh) (Remote host closed the connection) |
| 2026-03-07 12:13:14 | × | czan quits (~czan@user/mange) (Ping timeout: 245 seconds) |
| 2026-03-07 12:14:10 | × | merijn quits (~merijn@host-cl.cgnat-g.v4.dfn.nl) (Ping timeout: 248 seconds) |
| 2026-03-07 12:19:46 | → | target_i joins (~target_i@user/target-i/x-6023099) |
| 2026-03-07 12:23:39 | → | merijn joins (~merijn@62.45.136.136) |
| 2026-03-07 12:26:35 | → | Guest89 joins (~Guest89@185.45.21.144) |
| 2026-03-07 12:27:09 | <Guest89> | sorry I think I lost connection; I posted some replies to you sm but I'm not sure if they got through? |
| 2026-03-07 12:28:10 | <int-e> | no, but also: <+haskellbridge> <sm> got to go.. good luck |
| 2026-03-07 12:28:16 | × | merijn quits (~merijn@62.45.136.136) (Ping timeout: 244 seconds) |
| 2026-03-07 12:28:20 | <Guest89> | darn |
| 2026-03-07 12:29:55 | <Guest89> | if anyone else has any suggestions in the meantime, this is how my comparators are defined for the busiest functions https://paste.tomsmeding.com/jSnEEitE |
All times are in UTC.